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Summary 


Problem 


Technological  literacy  is  an  area  of  increasing  concern  to  the  Navy.  As  computer 
technology  becomes  pervasive,  it  is  necessary  to  understand  why  some  individuals 
adapt  readily  to  machines  while  others  do  not.  We  need  to  know  more  about  the  skills 
that  form  the  basis  for  technological  literacy. 

Objective 

The  objective  of  this  research  is  to  identify  general  problem  solving  skills  that  dis- 
tinguish effective  computer  problem  solvers  from  less  effective  ones. 

Approach 

Subjects  were  given  a  picture  drawing  task  using  MacPaint  and  an  Apple  Macin- 
tosh. Subjects  were  given  minimal  instruction  in  the  use  of  the  system.  To  complete 
the  task,  they  had  to  explore  the  system  to  learn  to  use  the  necessary  features.  A  pre- 
test and  posttest  questionnaire  were  used  to  determine  the  subjects'  skills  and  back- 
grounds. The  exploration  sessions  were  videotaped  using  a  split  screen  system  that  al- 
lowed observation  of  both  subject  and  computer  screen. 

The  subjects  ranged  from  an  eight  year  old  girl  to  a  sixty-eight  year  old  man. 
Some  subjects  had  experience  with  computers  or  word  processors.  None  had  any  prior 
experience  with  a  Macintosh. 

Results  and  Conclusions 

Contrary'  to  our  expectations,  mental  imagery  or  advance  organizers  seemed  to 
have  little  to  do  with  success  at  the  problem  solving  task.  There  seemed  to  be  two 
principal  determiners  of  success.  The  first  was  the  way  the  subject  chose  to  constrain 
the  task.  The  second  was  the  level  of  general  problem  solving  skill  that  the  subject 
brought  to  the  task 

Future  Research  Considerations 

A  future  study  should  break  subjects  into  various  groups  of  interest.  Such  groups 
would  include  grouping  by  age,  sex,  and  prior  background.  Within  these  target  groups, 
try  to  develop  taxonomies  of  problem  solving  styles.   These  could  include  willingness  to 


explore,  different  approaches  to  exploration  and  so  on.  Once  these  have  been  devel- 
oped, testing  could  begin  on  a  variety  of  experimental  treatments  lor  teaching  explora- 
tion or  technological  literacy  skills. 
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Introduction 


One  puzzle  that  one  notices  in  teaching  beginning  computer  courses  is  the  puzzle 
of  technological  literacy.  Some  students  seem  to  grasp  the  concepts  immediately. 
They  have  little  trouble  fitting  things  together  and  generalizing  their  discoveries.  For 
other  students,  the  whole  thing  is  an  exercise  in  frustration.  They  never  seem  to  be 
able  to  coordinate  their  knowledge.  Each  problem  is  viewed  as  unrelated  to  anything 
else.  The  whole  computer  ellort  is  seen  as  confusing  and  frustrating.  These  students 
are  never  able  to  get  things  together. 

This  is  sometimes  cited  as  evidence  for  the  need  to  teach  technological  literacy. 
The  world  is  becoming  more  dependent  on  technology,  and  what  is  needed  is  a  tech- 
nologically literate  citizenry  who  can  cope  with  this  new  world.  But  as  a  teacher,  one 
sometimes  notices  surprising  things  about  this  technological  literacy.  Occasionally  it 
happens  that  the  struggling  student  is  a  technologically  literate  engineer  while  the  stu- 
dent who  breezes  through  is  a  music  major  who  has  never  touched  a  computer  in  her 
life.   There  is  clearly  more  to  technological  literacy  than  simple  exposure  to  technology. 

In  the  first  research  report  on  general  problem  solving,  Lyons  and  Knott  [1984], 
identified  several  areas  as  candidates  for  further  research.  One  such  topic  is  in  the  in- 
tersection between  computer  science,  human  factors  and  psychology.  This  is  the  study 
of  exploration  and  exploratory  environments.  This  literature  is  represented  in  the  pa- 
pers by  Anzai  and  Simon  [1979],  Carroll,  et.  al.  [1982,  1982a,  1983,  1983a,  1984.  and 
1984a],  Darlington,  et.  al.  [1983],  Lewis,  et.  al.  11982],  Malone  [1981,  1981a.  1981b, 
1981c,  and  1982],  and  Schrager  and  Klahr  [1983].  This  literature  looks  at  the  way  peo- 
ple learn  about  an  environment  without  formal  instruction.  They  must  take  their  clues 
from  the  environment,  form  hypotheses  and  act  on  the  basis  of  these  hypotheses. 

This  study  is  of  interest  to  cognitive  psychologists  because  it  provides  a  way  to 
study  problem  solving  behavior  in  a  controllable  environment.  It  is  of  potentially 
greater  payoff  to  the  computer  industry.  The  man-machine  interface  is  a  make  or 
break  thing  for  a  computer  system.  In  microcomputers,  some  programs  are  popular 
partly  because  of  their  ease  of  use  (LOTUS  1-2-3  is  a  good  example).  Other  programs 
that  may  be  equally  powerful  but  unappealing  to  use  become  commercial  failures. 

This  work  with  exploratory  environments  investigates  how  people  solve  problems 
in  a  system  where  there  are  few  formal  clues  to  proper  behavior.  This  has  always  been 
a  feature  of  computer  systems.  The  manual  never  tells  you  everything  there  is  to  know 
about  a  system.  It  simply  cannot.  The  number  of  possible  combinations  in  a  dialog 
make    that  impossible.    It  has  always  been  true  that  some  people  are  inherently  better 


explorers  of  a  computer  environment  than  others.  But  until  recently,  there  has  been  no 
research  into  why  this  might  be  so.  Microcomputers  have  made  this  a  more  pressing 
question.  Surprisingly,  video  games  have  had  an  influence  as  well.  The  video  games 
show  that  exploration  of  complex  environments  is  feasible  and  even  a  reasonable  thing 
to  do.  There  are  few  manuals  on  video  games.  T he  widespread  use  of  micros  even  by 
non-technical  individuals  makes  a  better  understanding  of  exploration  necessary. 

The  work  on  exploration  in  the  computer  science  references  is  interested  primarily 
in  the  interface  design  question.  Examples  of  this  literature  include  the  papers  bv 
Brown  [1983],  Good,  Whiteside,  Wixon  and  Jones  [1984],  Jacob  [1983|,  Morland  [1983], 
Norman  [1983]  and  Stohr  and  White  [  1 982 j.  But,  there  is  a  flip  side  to  this  question 
that  is  of  interest  in  this  research.  How  can  we  teach  people  to  be  better  explorers?  Is 
this  a  teachable  skill?  Why  are  some  people  better  explorers  than  others?  This  is  the 
same  question  (applied  to  a  narrower  domain)  that  motivated  our  earlier  study.  As 
computers  become  more  pervasive,  these  are  questions  that  need  further  research. 


Relation  of  Problem  Solving  to  Navy  Needs 


The  issues  grouped  under  the  term  technological  literacy  are  becoming  increasingly 
important  to  the  Navy.  Computer  technology  affects  every  aspect  of  the  Navy  from 
offices  to  weapons  systems.  Because  of  the  increasing  cost  of  manpower,  the  push  is 
on  to  automate  office  operations.  The  government  cannot  afford  to  operate  with  ex- 
pensive manual  systems.  Manual  operations  and  paper  based  systems  are  giving  way  to 
office  automation. 

At  the  management  level,  there  are  computerized  decision  support  systems  to  aid 
in  budgeting  and  planning.  More  and  more  people  are  becoming  users  and  supervisors 
of  computer  systems.  In  the  tactical  Navy  computers  are  making  their  impact  felt  as 
well.  C3  and  intelligence  systems  rely  increasingly  on  computers  to  function.  Human 
reaction  times  are  too  slow  for  modern  warfare.  The  tactical  environment  is  becoming 
increasingly  computerized. 

It  is  unlikely  then  that  any  Naval  career,  be  it  officer  or  enlisted  will  be  untouched 
by  computers.  The  challenge  over  the  next  decade  will  be  to  integrate  computer  edu- 
cation and  computer  literacy  into  the  training  programs  of  the  Navy.  Without  this,  we 
will  not  meet  the  needs  for  technological  literacy  in  Naval  personnel. 

To  do  this,  however,  we  need  to  understand  the  components  of  technological  liter- 
acy. This  is  not  an  area  that  can  be  studied  using  conventional  structured  testing 
methodologies.  Measures  of  technological  literacy  are  harder  to  construct  than  they 
are  in  conventional  areas  like  reading.  We  are  not  interested  so  much  in  testing  con- 
ventional skills  as  we  are  in  testing  interest  and  attitudes.  We  need  to  know  the  skills  a 
person  brings  to  a  job  and  how  they  affect  abilities  to  do  job.  These  studies  should 
help  to  design  training  programs  to  meet  Navy  needs. 
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Other  Research  in  Exploration 


The  research  related  to  exploration  and  problem  solving  is  diverse  and  scattered.  It 
may  be  found  in  Computer  Science,  Cognitive  Psychology,  Educational  Psychology, 
and  Engineering.  There  is  no  one  discipline  where  this  literature  is  concentrated, 
found.  There  is  also  no  one  body  of  scientists  or  academicians  who  are  interested  in 
looking  into  this. 

Because  it  is  an  interdisciplinary  study,  the  literature  can  be  hard  to  track  and  dif- 
ficult to  understand.  In  doing  the  literature  survey  for  this  research,  I  have  divided  the 
literature  into  three  areas.  The  first  is  research  related  to  problems  solving,  the  second 
is  purely  exploration  oriented  research,  and  the  third  is  research  related  to  the  interface 
between  computer  science  and  psychology. 


Research  Related  to  Problem  Solving 

There  is  a  small  literature  in  educational  research  in  the  study  of  problem  solving. 
It  does  not  seem  to  be  an  area  that  has  attracted  widespread  interest  when  compared 
with  the  literature  on  more  conventional  subjects  like  mathematics  or  reading.  One 
article  by  Post  and  Brennan  [1976]  dealt  with  an  experiment  in  teaching  geometry  to 
two  groups  of  tenth  grade  students. 

The  authors  developed  a  test  of  problem  solving  ability  and  administered  it  as  both 
a  pretest  and  a  posttest  to  two  groups  of  tenth  graders.  One  group  had  received  in- 
struction in  the  General  Heuristic  Problem  Solving  Procedure  (presented  in  Figure  1) 
while  the  other  did  not.  The  two  groups  did  not  show  any  significant  difference  in 
problem  solving  ability.  The  authors  offer  a  detailed  discussion  of  the  statistical  meth- 
odology they  used.  This  discussion  would  be  appropriate  for  a  beginning  experimental 
methods  course.  Unfortunately,  they  tell  us  nothing  about  how  the  problem  solving 
methodology  was  taught.  Were  students  merely  exposed  to  it?  Did  they  get  substantial 
practice  with  it?  These  are  questions  left  unanswered.  The  conclusions  in  the  article 
are  too  strong  to  be  justified  by  the  material  presented. 

Raaheim  and  Kaufmann  [1974]  tested  73  secondary  school  students  using  tests  of 
the  subjects'  ability  to  manipulate  their  environment.  In  one  test,  the  subjects  were 
asked  to  tell  how  to  save  a  kitten  stranded  at  the  top  of  a  tree.  The  lowest  branch  on 
the  tree  was  too  high  to  reach.  The  tools  that  they  had  available  were  a  strong  drill,  a 


I.  Recognition,  Clarification  and  Understanding  of  the  Problem. 

1.  Read  the  problem  carefully. 

2.  Look  up  any  words  you  do  not  understand. 

3.  What  is  the  unknown?   What  are  the  data?    What  is  the 
condition?    What  is  given? 

4.  State  the  problem  in  your  own  words. 

5.  Break  the  problem  into  parts. 

6.  Draw  a  diagram  to  aid  in  clarification. 

7.  Accept  or  reject  the  specific  problem  tools  as  a  problem 
for  yourself. 

II.  Plan  of  Attack  -  Analysis 

1.  Gather  data  (facts,  rules,  relationships)  which  are 
necessary  for  solution. 

2.  Recall  missing  data,  select  relevant  data  from  the 
problem  statement,  and  generate  new  data  if  necessary. 

3.  Eliminate  irrelevant  data. 

4.  Decide  on  needed  approach  activities  by  noting  obstacles 
to  the  solution  of  the  problem. 

III.  Productive  Phase 

1.  Find  the  connection  between  the  data  and  the  unknown. 
You  may  have  to  consider  auxiliary  problems  if  an 
immediate  connection  cannot  be  found.   Do  you  know  a 
related  problem? 

2.  Generate  a  hypothesis  or  a  number  of  alternative 
hypotheses  (possible  solutions  of  the  problem). 

3.  Order  your  data  in  preparation  for  hypothesis  testing. 

4.  Reject  initial  hypotheses  that  do  not  satisfy  the 
conditions  of  the  problem. 

5.  Select  a  remaining  hypothesis  for  testing. 

6.  Construct  an  algorithm  or  develop  a  heuristic  for  the 
manipulation  of  data  as  an  instrument  for  possible 
verification  of  a  hypothesis. 

IV.  Validating  Phase  -  Checking  -  Proving 

1.  Accept  or  reject  the  hypothesis  by  verifying  or  not 
verifying  that  it  meets  the  conditions  of  the  task. 

2.  Look  back.   Can  you  check  the  result?   Can  you  check  the 
argument?   Can  you  derive  the  result  differently?   Can 
you  use  the  result  or  the  method  for  some  other  problem? 

3.  If  you  have  rejected  your  hypothesis,  select  a  remaining 
one  for  testing. 


Figure  1:  General  Heuristic  Problem  Solving  Procedure  -  Post 


hammer,  a  pair  of  pliers  and  a  plane.  The  "best"  solution  involved  drilling  holes  in  the 
tree  trunk  and  then  using  the  hammer,  pliers  and  even  the  drill  as  "steps"  to  climb  the 
tree.  There  were  other  tests  in  a  similar  vein.  They  seemed  to  test  the  subjects'  ability 
to  overcome  functional  fixity  in  solving  problems. 

Success  on  their  tests  correlated  highly  with  IQ.  Girls  were  significantly  poorer 
problem  solvers  than  boys.  The  authors  attributed  this  to  their  unfamiliarity  with  the 
tools  posed  in  the  problems.  A  possible  alternate  explanation  might  be  that  girls  are 
more  strongly  socialized  to  be  "good".  This  would  tend  to  cause  them  to  ignore  the 
rule  breaking  required  in  these  problems.  The  authors  conclude  that  general  problem 
solving  ability  exists  and  is  highly  correlated  with  conventional  measures  of  intelligence. 

The  paper  by  Reif  and  Heller  [1982]  is  a  prescriptive  piece.  They  lay  out  a  teach- 
ing methodology  for  physics  based  on  the  general  problem  findings  in  cognitive  psy- 
chology. They  outline  a  strategy  for  structuring  knowledge  and  acquiring  knowledge  in 
physics.  They  present  a  detailed  program  for  physics  instruction.  There  is  no  empiri- 
cal validation,  but  they  do  try  to  tie  the  points  stated  in  their  recommendations  into 
previous  empirical  work. 

A  paper  by  Baird  [1983]  offers  an  interesting  summary  of  the  problem  solving  re- 
search in  education.  Baird's  report  is  similar  to  ours  in  that  he  is  surveying  the  litera- 
ture to  see  what  has  been  done  in  problem  solving  and  what  benefits  may  result  from 
teaching  problem  solving.  He  concludes  that  the  benefits  from  teaching  generic  prob- 
lem solving  are  difficult  to  prove.  He  believes  that  general  problem  solving  techniques 
are  probably  most  effective  when  they  are  integrated  into  domain  specific  courses. 

A  last  paper  is  that  by  Ruth  Pitt  [  1 983 1.  It  is  a  comprehensive  literature  survey  of 
the  psychological  literature  on  the  development  of  general  problem  solving  schema. 
The  work  concludes  that  the  development  of  a  GPS  schema  takes  place  between  ado- 
lescence and  early  adulthood.  She  studied  two  groups  of  students,  one  group  of  tenth 
graders  and  one  group  of  college  students.  She  found  that  the  tenth  graders  had  limit- 
ed ability  to  define  problems  and  generate  hypotheses.  They  did  not  seem  to  have  a 
rich  set  of  problem  representations  or  methods  for  dealing  with  problems.  The  college 
students  were  much  better.  Their  abilities  approached  that  of  professors  on  most 
measures.  The  findings  here  are  preliminary,  but  there  are  some  interesting  implica- 
tions. Pitt  suggests  that  the  time  to  teach  general  problem  solving  techniques  is  in 
early  adolescence  when  such  concepts  are  being  formed.  More  mature  students  are 
likely  to  respond  better  to  domain  specific  techniques.  For  better  or  worse,  their  skills 
may  already  be  formed. 

If  these  papers  are  indicative  of  the  general  level  of  interest  in  education  in  teach- 
ing general  problem  solving,  then  the  level  of  interest  cannot  be  very  high.  It  would 
seem  that  the  primary  influence  of  the  general  problem  solving  work  has  been  to  stim- 
ulate new  approaches  to  the  teaching  of  conventional  disciplines  as  in  the  Reif  and 
Heller  article.  There  is  little  work  involving  the  teaching  or  study  of  general  problem 
solving  ability.    It  may  be  that  the  concept  is  too  hard  to  define  and  that  it  is  safer  to 


stay  with  conventional  disciplines.  It  may  also  be  that  the  best  approach  to  general 
problem  solving  techniques  is  to  integrate  them  in  conventional  areas  to  improve  re- 
tention and  comprehension. 


Exploration  Research 

An  interesting  literature  studying  problem  solving  is  growing  in  the  intersection 
between  computer  science,  human  factors  and  psychology.  This  is  the  study  of  explo- 
ration and  exploratory  environments.  This  literature  is  represented  in  the  papers  by 
Anzai  and  Simon  [1979],  Carroll,  et.  al.  [1982,  1982a,  1983,  1983a,  1984,  and  1984a], 
Darlington,  et.  al.  [1983],  Lewis,  et.  al.  [1982],  Malone  [1981,  1981a,  1981b,  1981c,  and 
1982),  and  Schrager  and  Klahr  [1983].  This  literature  looks  at  the  way  people  learn 
about  an  environment  without  formal  instruction.  They  must  take  their  clues  from  the 
environment,  form  hypotheses  and  act  on  the  basis  of  these  hypotheses. 

This  study  is  of  interest  to  cognitive  psychologists  because  it  provides  a  way  to 
study  human  problem  solving  behavior  in  a  controllable  environment.  It  is  of  poten- 
tially greater  payoff  to  the  computer  industry.  The  quality  of  the  man-machine  inter- 
face can  be  crucial  to  the  success  of  a  piece  of  software.  In  microcomputing,  some 
programs  are  popular  partly  because  of  their  ease  of  use  (LOTUS  1-2-3  is  a  good  ex- 
ample). Other  programs  that  may  be  equally  powerful  but  unappealing  to  use  become 
commercial  failures. 

Carroll  [1982a]  compares  learning  to  use  a  word  processor  with  the  fantasy  game 
"Adventure".  Both  are  computer  programs.  The  Adventure  program  is  intentionally 
confusing,  but  it  draws  users  into  the  world  it  creates.  In  Carroll's  comparison,  many 
word  processors  offer  the  same  kind  of  confusion  as  Adventure  without  olfering  a  bal- 
ancing incentive  to  push  past  the  confusion  and  learn  the  system. 

The  problems  noted  by  Carroll  can  be  formidable  obstacles  in  learning  to  use  a 
computer  system.  Some  people  regard  them  as  so  intimidating  that  they  are  never  able 
to  master  the  machine.  Others  regard  them  as  a  challenge  and  even  become  compul- 
sive about  responding  to  it.  Carroll  offers  a  series  of  suggestions  for  system  design  to 
reduce  the  problems  noted  in  Figure  1.  These  suggestions  are  given  in  Figure  2.  Car- 
roll regards  his  work  as  preliminary.  These  results  were  published  in  the  IEEE  Com- 
puter, and  seemed  generally  interested  in  stimulating  thought  on  software  design  issues. 

Malone  [1981]  is  studying  the  same  kind  of  issues  as  Carroll.  His  work  also  focus- 
es on  the  relationship  between  computer  games  and  more  conventional  computer  pro- 
grams. He  is  interested  in  what  makes  games  so  intriguing  and  whether  these  features 
of  the  game  interface  could  be  transferred  to  other  programs.  He  ran  subjects  through 
a  series  of  learning  game  experiments  and  tried  to  abstract  features  of  the  game  that 
worked  best.  From  this,  he  developed  a  set  of  heuristics  for  designing  enjoyable  inter- 
faces.  These  are  given  in  Figure  3. 


Responsiveness 
Benchmarks 


Acceptable  uncertainty 

Safe  Conduct 
Learning  by  doing 

Opportunity 

Taking  charge 
Control 


When  you  do  something,  you  get  some 
feedback  (at  least  informational). 

You  can  tell  where  you  are  within  a 
given  episode  or  session.   You  have  the 
means  for  assessing  achievement  and 
development  of  skill. 

Being  less  than  fully  confident  of  your 
understanding  and  expertise  is  OK. 

You  cannot  do  anything  too  wrong. 

You  do  so  that  you  can  learn  to  do;  you 
design  a  plan;  you  do  not  merely  follow 
a  recipe. 

Most  of  the  things  you  learn  to  do  work 
everywhere.  You  can  reason  out  how  to 
do  many  other  things. 

If  progress  stagnates,  something  new  is 
suggested  or  happens  spontaneously. 

You  are  in  control  or  at  least  have  the 
illusion  of  being  in  control. 


Figure  2: 


Exploratory  Environment  Characteristics  -   Carroll[  1982a] 


This  work  with  exploratory  environments  is  trying  to  find  out  how  people  solve 
problems  in  a  system  where  there  are  few  formal  clues  to  proper  behavior.  This  has 
always  been  a  feature  of  computer  systems.  The  manual  never  tells  you  everything 
there  is  to  know  about  a  system.  It  simply  cannot.  The  combinatorics  render  that  im- 
possible. It  has  always  been  true  that  some  people  are  inherently  better  explorers  of 
this  computer  environment  than  others.  But  until  recently,  there  has  been  no  research 
into  why  this  might  be  so.  Microcomputers  and  video  games  have  made  this  a  more 
pressing  question.  The  video  games  show  that  exploration  of  complex  environments  is 
feasible  and  even  a  reasonable  thing  to  do.  There  are  few  manuals  to  video  games. 
The  widespread  use  of  micros  even  by  non-technical  individuals  makes  a  better  under- 
standing of  exploration  necessary. 

The  work  on  exploration  in  the  references  is  interested  primarily  in  the  interface 
design  question.    But,  there  is  a  flip  side  to  this  that  is  of  interest  in  this  research.    How 


I.   Challenge 


A.  Goal.    Is  there  a  clear  goal  in  the  activity?    Does  the 
interface  provide  performance  feedback  about  how  close 
the  user  is  to  achieving  his  goal? 

B.  Uncertain  outcome.    Is  the  outcome  of  reaching  the  goal 
uncertain? 

1.  Does  the  activity  have  a  variable  difficulty  level? 
For  example,  does  the  interface  have  successive 
layers  of  complexity? 

2.  Does  the  activity  have  multiple  level  goals?    For 
example,  does  the  interface  include  scorekeeping? 


II.    Fantasy 


A.  Does  the  interface  embody  emotionally  appealing 

fantasies? 

B.  Does  the  interface  embody  metaphors  with  physical  or 

other  systems  that  the  user  already  understands? 


III.   Curiosity 


A.  Does  the  activity  provide  an  optimal  level  of 
informational  complexity? 

1.  Does  the  interface  use  audio  and  visual  effects: 
(a)  as  decoration,  (b)  to  enhance  fantasy  and 
(c)  as  a  representation  system? 

2.  Does  the  interlace  use  randomness  in  a  way  that  adds 
variety  without  making  tools  unreliable? 

3.  Does  the  interface  use  humor  appropriately? 

B.  Does  the  interface  capitalize  on  the  users'  desire  to 
have  "well-formed"  knowledge  structures?    Does  it 
introduce  new  information  when  users  see  that  their 
existing  knowledge  is  (1)  incomplete,  (2)  inconsistent 
or  (3)  unparsimonious? 


Figure  3:    Interface  Design  Heuristics  -  Malone  [1981] 
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can  we  teach  people  to  be  better  explorers?  Is  this  a  teachable  skill?  This  is  the  same 
question  (applied  to  a  narrower  domain)  that  motivates  this  whole  report.  As  com- 
puters become  more  pervasive,  this  is  a  question  that  deserves  research. 


Psychology  and  Computer  Programming 

There  is  a  large  literature  in  computer  science  and  psychology  that  relates  to  computer 
programming  and  human  factors.  Only  a  small  fraction  of  this  literature  is  relevant  to 
our  study.  Much  of  this  literature  concerns  the  way  in  the  way  people  interact  with 
complicated  systems.  This  interaction  may  be  by  experts  who  are  well  educated  in  the 
technology  of  this  system  involved.  Frequently  these  studies  revolve  around  way  in 
which  to  structure  these  systems  so  that  experts  can  use  them  more  effectively.  This  is 
the  general  focus  of  the  human  factors  literature  as  it  revolves  around  such  topics  as 
the  design  o[  aircraft  cockpits  so  that  pilots  can  use  them  effectively  in  stressful  situ- 
ations. 

The  Computer  Science  related  literature  is  similar  in  that  it  deals  with  the  difficul- 
ties involved  in  using  certain  classes  of  programming  languages.  The  normal  focus  in 
both  classes  of  literature  is  either  design  of  systems  for  specialists,  or  the  selection  of 
individuals  for  training  as  specialists. 

Neither  of  these  focuses  was  of  interest  for  our  study.  We  were  more  interested  in 
the  way  in  which  novices  or  relative  novices  would  react  in  an  exploratory  environ- 
ment. We  wanted  to  deal  with  an  environment  that  was  unfamiliar  to  the  subject. 
This  meant  that  we  could  make  very  few  assumptions  about  the  abilities  of  the  sub- 
jects. 

In  addition,  we  had  only  a  limited  time  in  which  to  deal  with  the  subjects.  This 
meant  that  we  could  not  give  them  a  complicated  set  of  instructions  as  would  be  typi- 
cal in  a  computer  programming  class.  Hence,  there  was  no  actual  programming  in- 
volved in  our  experiments.  Because  of  this  focus  of  interest  of  ours,  a  much  narrower 
subset  of  the  diverse  literature  was  actual  relevant  to  our  study.  Two  areas  were  of  pri- 
mary interest,  the  first  was  mental  imagery,  and  the  second  was  the  educational  psy- 
chology literature  dealing  with  the  subject  of  teaching  computer  programming. 

Mental  imagery  deals  with  the  images  that  people  use  to  represent  information. 
We  were  interested  in  mental  imagery,  because  we  wanted  to  find  out  if  there  were  a 
set  of  mental  images  that  were  useful  in  organizing  an  exploratory  process.  There  is  no 
hard  evidence,  but  there  is  folklore  in  the  computer  area  that  certain  types  of  people 
are  better  disposed  in  learning  computing  than  others.  One  example  of  folklore  is  the 
statement  sometimes  heard  is  that  musicians  make  good  computer  programmers. 

If  this  is  true,  then  perhaps  music  offers  a  set  of  compatible  mental  images  that  aid 
in  the  learning  of  computer  topics.    Mental  imagery  is  discussed  in  a  series  of  articles 
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by  Kosslyn  [1981],  and  by  Kosslyn,  Pinker,  Smith,  Shwartz  [1979].  The  latter  article 
presents  a  theory  of  mental  imagery.  It  outlines  the  general  research  directions  taken 
and  provides  an  overview  of  the  empirical  foundations  of  mental  imagery.  This  article 
presented  the  operating  basis  for  the  mental  imagery  concept  that  we  are  trying  to  an- 
alyze in  this  research. 

The  educational  psychology  literature  was  much  richer.  In  a  series  of  papers,  R. 
E.  Mayer  [1975,  1976,  1981]  deals  with  the  problems  of  learning  a  computer  language. 
In  the  1975  paper,  the  study  explored  how  novices  learn  to  interact  with  a  computer. 
The  research  goal  was  to  develop  instructional  guidance  to  aid  beginners  in  using  the 
computer  effectively. 

Mayer  looked  at  how  a  beginner  acquires  knowledge  about  the  computer.  He  saw 
the  user  as  bringing  a  mental  model  to  the  learning  situation.  The  model  is  used  as  a 
vehicle  for  learning  new  material.  Mayer  is  primarily  concerned  with  the  effect  of  dif- 
ferent learning  sets  (cognitive  structure,  mental  models,  prerequisite  experience...)  on 
the  acquisition  of  computer  skills.  He  experimentally  manipulates  the  activation  of 
mental  models  used  by  novice  computer  users  by  providing  different  instructional  sets. 

Three  experiments  were  run.  In  all  experiments  subjects  were  tested  by  one  of  two 
methods.  The  generation  test  required  subject  to  write  a  program  from  a  stimulus 
statement.  In  the  interpretation  test  subjects  were  asked  to  describe  what  the  program 
accomplished. 

The  hypothesis  tested  was  whether  two  mental  models  of  how  a  computer  system 
operated  influenced  the  programming  knowledge  acquired.  One  learning  set  was  a  dia- 
gram model  which  contained  familiar  objects  and  activity  as  an  analog  to  the  comput- 
er. The  other  learning  set  was  a  flow  chart  system  of  geometric  symbols  of  the  com- 
puter system.  The  two  learning  sets  differed  in  the  amount  of  related  knowledge  the 
models  contained  from  the  subjects'  own  experience.  They  used  a  simplified  version  of 
FORTRAN. 

Subjects  receiving  the  diagram  model  performed  better  than  those  who  did  not  on 
the  interpretation  test.  The  flowchart  subjects  performed  better  on  the  generation  test. 
He  also  observed  that  the  model  seemed  to  help  low  ability  subjects'  performance  more 
than  it  did  the  high  ability  subjects. 

Mayer  concluded  that  learning  technical  material  is  easier  when  the  learner  is  given 
a  model  of  the  overall  task.  The  model  should  be  stated  in  familiar  terms,  understand. 
New  material  may  be  assimilated  and  organized  more  easily  than  without  a  model.  He 
terms  these  models  as  "advance  organizers."  Since  the  novice  users  in  his  experiment 
showed  qualitative  differences  in  performance  depending  on  the  type  of  model  used, 
Mayer  concludes  that  some  advance  organizers  are  better  than  others  and  that  it  is  re- 
lated to  the  degree  of  model  meaningfulness  to  the  subject. 
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Mayer  reported  further  on  this  work  in  1976  in  "Some  conditions  of  meaningful 
learning  for  computer  programming:  Advance  organizers  and  subject  control  of  frame 
order."  He  defines  meaningful  learning  as  receiving  information,  having  a  cognitive 
structure  available  to  hold  information,  and  actively  processing  the  material. 

In  this  study,  he  predicts  that  the  amount  of  information  encoded  when  learning 
new  technical  information  will  increase  when 

1.  The  learner  has  a  meaningful  mental  model  of  the  what  is  to  be  learned. 

2.  The  new  material  is  actively  processed  during  the  learning  phase. 

He  further  predicts  that  when  these  conditions  are  met,  there  will  be  a  greater  transfer 
of  knowledge  to  similar  problem  domains.  Mayer  designed  two  experiments  to  look  at 
the  information  learned  and  how  it  is  structured.  The  experimental  conditions  involved 
the  presence  or  absence  of  meaningful  learning  sets  before,  during,  and  after  learning  a 
computer  programming  language. 

Nonprogrammer  subjects  were  taught  a  simplified  version  of  FORTRAN.  The 
subjects  were  tested  in  one  of  two  ways: 

1.  Generation  test  questions  requiring  correct  language  statements  in  the  cor- 
rect order. 

2.  Interpretation  test  questions  requiring  correct  interpretion  of  language  state- 
ments. 

Two  experiments  were  run  with  much  the  same  results  as  the  earlier  study. 

In  1981,  Mayer  reviewed  research  concerned  with  the  learning  of  technical  infor- 
mation. He  i elates  this  to  his  interest  in  teaching  nonprogrammers  to  interact  with  the 
computer.  He  asserts  that  learning  is  facilitated  when  a  conceptual  framework  is  used 
to  connect  the  new  information  to  existing  knowledge.  He  bases  much  of  his  argument 
for  using  conceptual  models  on  Newell  and  Simon's  model  of  Human  Information 
Processing  and  on  Ausubel's  concept  of  advance  organizers. 

He  makes  the  distinction  between  rote  and  meaningful  learning.  Meaningful 
learning  involves  the  connection  between  the  existing  knowledge  in  long-term  memory 
and  the  recently  acquired  information  in  short-term  memory.  Material  learned  by  rote 
simply  exist  in  the  memories  as  isolated  units  of  information  with  little  or  no  connec- 
tion with  the  existing  knowledge  structures  in  long-term  memory.  Meaningful  learning 
facilitates  understanding  which  in  turn  allows  one  to  apply  what  is  learned  across  a 
spectrum  of  situations  similar  to  the  circumstances  under  which  it  was  originally 
learned. 
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He  emphasizes  meaningful  learning  as  the  ability  to  transfer  learned  knowledge  to 
new  situations.     Meaningful  learning  occurs  under  the  following  conditions: 

1.  Reception  -  the  learner  must  attend  to  the  information  to  be  learned  in  order 
to  receive  the  information  in  short-term  memory. 

2.  Availability  -  the  learner  must  have  an  existing  knowledge  structure  in  long- 
term  memory  to  process  the  information  received  in  short-term  memory. 

3.  Activation  -  the  learner  must  actively  use  the  concepts  from  long-term  mem- 
ory during  learning  to  establish  the  connection  of  the  new  information. 

Much  of  his  research  addresses  the  question  of  whether  the  learning  and  transfer 
of  technical  information  is  increased  by  providing  the  learner  with  a  familiar  organiza- 
tion plan  for  the  new  material.  He  presents  evidence  from  other  fields  to  support  his 
approach.  The  idea  is  to  provide  an  introduction  that  contains  only  the  general  ideas  to 
be  presented  in  the  new  material. 

From  the  evidence  presented  it  was  concluded  that: 

1.  To  assimilate  new  material  the  learner  must  have  some  type  of  conceptual 
framework  available  during  learning. 

2.  Studies  show  an  increase  in  the  amount  of  material  recalled  amd  learned 
when  the  learner  is  given  a  meaningful  conceptual  framework  during  learn- 
ing. The  conceptual  framework  acts  as  an  advance  organizer  for  new  ma- 
terial to  be  incorporated  into  the  learner's  existing  knowledge. 

3.  The  advance  organizers  are  most  effective  when: 

a)  Learning  unfamiliar  material. 

b)  Used  writh  low-ability  subjects. 

c)  Testing  transfer  of  knowledge  to  new  situations. 

4.  Generating  useful  advance  organizers  requires: 

a)  Logical  relations  between  the  ideas  presented  in  the  material. 

b)  Logical  connections  between  the  new  material  and  the  model  containing 
familiar  concepts. 

Mayer  discusses  the  use  of  concrete  models  in  learning  computer  programming. 
Concrete  models  have  been  used  in  other  studies  to  increase  technical  comprehension. 
A  concrete  model  is  a  representation  which  is  an  analogy  for  the  concepts    in  the  ma- 
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terial  to  be  learned.  A  concrete  model  is  tangible  representation.  It  should  ensure  that 
the  learner  has  a  familiar  conceptual  structure.  lie  experiments  with  concrete  models 
of  computer  programming  as  advance  organizers  for  nonprogrammers  to  use  before, 
during  and  after  learning  a  new  programming  language.  He  reviews  his  studies  with 
concrete  models  and  concludes: 

1.  Concrete  models  enhance  transfer  performance  but  seems  not  to  affect  sim- 
ple retention  of  new  material. 

2.  Concrete  models  are  useful  when  introduced  before  and  during  learning. 
They  are  not  useful  afterwards  because  the  subjects  have  already  encoded 
material  in  rote  manner. 

3.  Concrete  models  enhance  ability  to  recall  conceptual  ideas  in  the  newly 
learned  computer  languages. 

4.  Concrete  models  improve  low-ability  subjects  the  most.  High-ability  subjects 
may  already  possess  a  good  conceptual  model. 

A  related  survey  paper  is  the  paper  by  Moran  [1981],  which  presents  an  overview 
of  the  field  of  user  psychology.  This  is  an  attempt  to  develop  a  psychology  of  user  be- 
havior. Research  focuses  on  the  cognitive  aspects  of  the  user  interface  in  specific  tasks. 
He  believes  that  the  usefulness  of  a  system  ultimately  depends  on  the  quality  of  the 
user-computer  interaction. 

He  lists  the  components  that  determine  user  behavior  as: 

1.  User's  goal  -  how  one  organizes  actions  to  reach  goal. 

2.  Task  structure-  the  rules  (boundaries)  of  the  computer  system,  language,  etc. 
This  is  often  viewed  as  the  "user  interface." 

3.  User's  knowledge-  knowledge  of  the  rules  (task). 

4.  User's  processing  limits  -  which  strategies  are  adopted  are  determined  by  the 
limits  of  ability  to  process  the  information,  i.e.  Short-term  and  long  term 
memory,  ways  to  relieve  memory  load. 

He  believes  we  can  improve  user  behavior  by: 

1.  Altering  the  interface. 

2.  Offering  instruction  to  the  user. 

3.  Changing  the  user  goals. 
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4.  Compensation  for  user  limitations. 
He  states  that  the  dimensions  of  user  behavior  must  be  evaluated  by: 

1.  Functionality 

2.  Learning 

3.  Time 

4.  Errors 

5.  Quality 

6.  Robustness 

7.  Acceptability 

We  should  describe  population  of  tasks  associated  with  particular  system  to  properly 
incorporate  the  user  in  the  total  design  of  the  system. 

Moran  believes  that  users  vary  along  two  primary  dimensions: 

1.  Amount  of  knowledge  -  this  usually  correlates  with  experience.  The  chal- 
lenges faced  by  novice  and  experts  users  are  different.  The  novice  user  is 
continuously  learning  more  about  the  system  to  solve  a  particular  computing 
problem.  The  expert  user  interacts  more  procedurally  to  achieve  a  goal  set 
by  the  computing  task.  The  expert  tends  to  be  using  a  highly  developed 
skill.  He  claims  performance  differences  marked  among  novices  using  differ- 
ent interfaces  but  performance  differences  seem  to  diminish  with  experience. 

2.  The  type  of  tasks  engaged  by  user  -  Expert  users  are  engaged  in  program- 
ming while  novices  are  simply  using  applications  programs.  Programming  in 
the  purest  sense  is  harder  to  learn  and  more  cognitively  complex  skill. 

Moran's  field  of  interest  is  the  user  interface.  This  is  any  interaction  between  the 
computer  and  the  user,  any  part  of  the  computer  system  that  the  user  comes  into  con- 
tact with.  It  can  be  the  physical  machine  or  the  perceptual  aspects  of  the  system.  An 
important  factor  of  the  interface  is  the  extent  to  which  the  internal  structures  of  the 
system  are  understood  by  the  user,  that  is,  the  user's  overall  conceptual  view  obtained 
from  the  behavior  of  the  computer  system.  This  is  termed  the  user's  conceptual  model 
of  how  the  system  operates.  This  conceptual  model  influences  how  the  user  will  or- 
ganize his  actions  to  complete  a  particular  computing  task.  This  is  similar  to  Mayer's 
advance  organizers. 
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There  have  been  several  studies  of  the  proper  approaches  to  studying  human-ma- 
chine interaction.  One  such  is  Brooks  [1980].  He  looks  at  use  of  behavioral  method- 
ologies to  describe  programming  language  features  and  cites  problems  with  current  re- 
search. He  feels  that  a  major  problem  is  usually  found  in  the  selection  of  subjects  and 
selection  of  materials. 

He  talks  about  the  use  of  open-ended  questions  as  way  to  assess  the  cognitive 
structure  a  programmer  may  have  in  a  computing  task.  Protocol  analysis  is  mentioned 
as  a  form  of  the  "open-ended  question"  measure.  He  recommends  that  the  most  fruitful 
approach  in  the  current  state  of  this  research  area  be  a  "successful  characterization  of 
the  programmer  interaction  ...  based  on  model  of  the  process  or  processes  used  by  a 
programmer  in  interacting  with  a  program.  The  development  of  the  cognitive  process- 
es involved  in  programming  is  therefore,  likely  to  be  a  prerequisite  to  progress  on  these 
methodological  issues". 

Another  paper  on  programmer  -  machine  interaction  is  by  Sheil  [1981].  He  shows 
that  many  accounts  constituting  the  basis  of  using  particular  programming  techniques 
are  based  informal  expert  opinion  in  computer  science  or  on  experimental  evidence 
which  is  methodologically  flawed.  He  emphasizes  importance  of  claims  which  describe 
a  psychological  basis  for  such  assertions  about  programming  methodology  but  also 
points  out  the  lack  of  empirically  testable  hypotheses  to  back  them  up. 

Sheil  asserts  that  many  behavioral  studies  of  programming  behavior  are  premature 
in  their  claims  to  be  measure  universal  underlying  processes  in  programming.  Since 
computing  behavior  is  a  highly  developed  skill  and  that  the  level  of  skill  is  dependent 
on  the  extent  of  the  programmer's  knowledge  base,  he  suggests  that  programming  be- 
havior first  be  characterized  using  more  introspective  techniques.  This  tends  to  sup- 
port the  use  of  protocol  analysis  as  a  first  step  in  identifying  the  user's  knowledge 
structure. 


Experimental  Setup 


Subjects 

A  variety  of  volunteer  subjects  participated  in  this  experiment.  The  group  included 
both  novice  computer  users  and  experienced  programmers.  The  novice  subjects  were 
mostly  volunteers  from  the  Administrative  Sciences  Department  stall'.  The  experienced 
subjects  were  volunteers  from  the  computer  center  at  the  Naval  Postgraduate  School. 
Three  children  were  also  included  among  the  subjects. 


Apparatus 

The  primary  experimental  apparatus  was  an  Macintosh  microcomputer  and  mouse 
interface.  The  microcomputer  included  a  keyboard  which  was  not  necessary  for  this 
experiment.  The  subjects  used  the  MacPaint  program  to  reproduce  an  image  on  the 
screen  of  the  Macintosh.  The  program  provides  a  screen  drawing  window  and  several 
drawing  tools.  MacPaint  represents  tools  by  icons  on  the  left  side  of  the  screen.  An 
icon  is  a  picture  that  represents  the  function  that  the  MacPaint  program  can  perform. 

The  user  selects  an  icon  by  moving  the  mouse  pointer  to  the  icon  and  clicking  the 
mouse  button.  The  subject  then  moves  the  mouse  pointer  to  the  screen  and  proceeds 
to  use  the  selected  function.  Subjects  could  then  produce  a  picture  by  using  the  select- 
ed icons  and  the  mouse  to  draw  on  the  screen. 

The  experiment  was  conducted  in  a  classroom  containing  a  two-way  mirror  and 
audio  visual  and  video  equipment.  Subjects  were  seated  at  a  table  in  front  of  the  Ma- 
cintosh computer.  The  table  had  ample  space  to  allow  free  movement  of  the  mouse. 
Two  television  cameras  were  used  to  record  protocols.  One  camera  was  to  the  side  and 
the  other  was  behind  the  subject.  They  did  not  interfere  with  the  task.  Judging  from 
the  reactions  during  filming,  the  subjects  did  not  seem  to  be  conscious  of  the  equip- 
ment. 

The  cameras  recorded  a  split  screen  image.  One  photographed  the  subject  and  the 
other  photographed  the  computer  screen.  In  this  way,  the  experimenters  could  observe 
the  images  as  they  evolved.  The  two  images  combined  on  video  tape  giving  a  complete 
record  of  the  subjects'  experiences.  In  addition,  the  drawings  produced  were  saved  to  a 
disc  and  were  also  printed  on  an  Imagewriter  printer. 
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Procedure 

The  subjects  were  given  a  picture  of  a  house  drawn  using  MacPaint  (see  Figure  4). 
The  subjects  were  instructed  to  reproduce  the  house  as  closely  as  possible.  The  house 
was  a  simple  drawing.  The  elements  of  the  drawing  were  chosen  to  exercise  as  many  of 
the  options  on  the  icons  of  the  Macintosh  as  possible.  The  elements  making  up  the 
house  ranged  from  simple  to  complex. 

Subjects  were  given  a  brief  description  of  the  computer  system  before  they  were 
turned  loose.  They  were  shown  how  to  select  icons  and  how  to  erase  the  screen  for 
backtracking.  The  subjects  were  encouraged  to  talk  freely  about  what  they  were  doing 
during  the  experiment.  Surprisingly,  few  people  chose  to.  They  seemed  to  become  en- 
grossed in  the  task  and  did  not  say  much  about  what  they  were  doing. 

During  the  testing  session  the  experimenter  monitored  the  session  from  behind  the 
subject  near  the  video  equipment.  The  subjects'  performance  was  evaluated  from  the 
video  tapes  of  the  experimental  session. 

The  experimental  methodology  used  in  these  studies  is  a  traditional  cognitive  psy- 
chology protocol  analysis.  The  study  of  human-machine  interaction  is  at  too.  early  a 
level  to  design  structured  tests.  Protocol  analysis  allows  observation  of  problem  solv- 
ing behavior.  We  use  this  to  assess  the  strategies  users  bring  to  a  computer  problem 
solving  situation. 

The  experiment  was  not  designed  to  assess  differences  in  aptitude.  Evidence  shows 
that  programming  aptitude  involves  large  knowledge  bases  and  is  hard  to  measure. 
Aptitude  depends  on  the  individual's  experience  [Sheil  1981;  Moran,  1981].  We  tried  to 
avoid  giving  an  individual  an  advantage  because  of  background.  MacPaint  and  the 
Macintosh  were  chosen  because  they  were  easy  to  use  with  no  experience. 

The  Macintosh  is  still  a  sufficiently  unusual  system  that  none  of  our  experienced 
subjects  had  used  one.  Further,  none  of  the  experience  factors  that  our  subjects  had 
applied  directly  to  the  Macintosh.  The  attitudes  and  background  of  the  subject 
brought  to  the  task  were  of  greater  interest.    Specific  skills  were  unimportant. 

Pretest  and  posttest  questionnaires  were  used  to  assess  the  experience  of  the  sub- 
jects (see  Appendix  B).  Of  special  interest  were  the  impact  of  advance  organizers 
[Mayer,  1981].  A  further  interest  was  the  cognitive  structures  or  mental  imagery  sub- 
jects brought  to  the  task.  In  planning  the  experiments  it  was  felt  that  mental  imagery 
or  advance  organizers  might  have  a  significant  effect  on  performance.  Some  of  the 
questions  in  the  pretest  were  designed  to  elicit  the  mental  imagery  that  the  subject 
might  use. 

In  computing,  there  is  much  folklore  but  little  hard  evidence  about  the  factors  that 
go  to  make  a  good  computer  user.  The  dimensions  of  technological  literacy  are  still 
largely  unexplored. 
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Figure  4:    Macintosh  House  Drawing  Problem 
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Before  we  can  teach  technological  literacy,  we  need  to  know  what  it  is.  Quantita- 
tive measures  will  not  help  if  we  do  not  know  what  to  measure.  Protocol  analysis  is  the 
best  choice  for  experimental  design  here. 


Discussion  of  the  Experiments 


The  house  drawing  Figure  4  is  a  simple  block  drawing.  The  simplicity  is  deceiv- 
ing, however.  In  Figure  5,  we  outline  the  house  features  and  the  tools  the  subject  must 
use  to  draw  the  house  features.  The  house  features  were  selected  to  force  the  subject 
to  explore  the  machine.  It  is  not  easily  possible  to  select  a  single  option,  stay  with  it, 
and  produce  the  house  in  Figure  4. 

All  subjects  started  with  the  walls  of  the  house.  This  feature  required  texture  and 
line  thickness  selection.  To  select  texture,  they  must  visit  the  menus  at  the  side  of  the 
Macintosh  screen.  To  select  line  thickness,  the  menu  at  the  lower  left  corner  is  re- 
quired. The  subject  selects  texture  using  the  menus  at  the  bottom  of  the  screen.  The 
filled  rectangle  icon  is  required  to  draw  the  walls.  This  is  selected  from  the  menu  at  the 
side  of  the  screen.  Another  approach  is  to  use  the  blank  rectangle  and  then-  use  the 
paint  pot  icon  for  texture.  None  of  our  subjects  use  this  approach,  although  one  used 
the  blank  rectangle  and  tried  to  color  it  in  with  the  pencil  icon. 

The  ridgeline  of  the  house  requires  the  line  thickness  selection  and  a  blank  rectan- 
gle. If  the  subjects  have  selected  a  filled  rectangle  previously,  they  must  reverse  that 
selection.  Must  subjects  either  did  not  notice  the  ridgeline  or  deemed  it  an  unnecessary 
detail.  The  small  window  required  a  line  thickness  selection,  the  blank  rectangle  icon, 
and  the  use  of  the  rubber  band  line  icon.  Many  of  the  subjects  did  not  made  transition 
from  filled  rectangle  to  blank  rectangle.  They  ended  up  drawing  a  filled  rectangle  and 
then  using  the  eraser  icon  to  erase  the  filling. 

The  picture  window  required  a  line  thickness  selection  and  the  use  of  the  blank 
rectangle  icon.  The  door  required  the  same.  The  door  window  required  a  blank  circle 
icon.  This  required  the  subjects  to  change  shapes.  The  door  knob  required  the  filled 
circle  icon.  This  meant  the  subjects  had  to  change  texture.  The  roof  required  a  texture 
and  line  thickness  selection.  The  subjects  had  to  select  a  filled  rubber  band  icon  to 
draw  the  irregular  shape  of  the  roof.  An  alternate  approach  to  this  would  be  to  select 
the  paint  pot  and  use  blank  rubber  band  shapes. 

The  chimney  required  texture  and  line  thickness  selections.  This  meant  that  using 
the  filled  rectangle  icon.  An  alternative  would  be  to  use  a  blank  rectangle  and  the 
paint  pot  icon.  The  smoke  coming  out  of  the  chimney  required  the  use  of  the  pull- 
down menu. 

The  pull-down  menu  seemed  to  be  the  hardest  feature  for  subjects  to  find.  This  is 
the  set  of  selections  at  the  top  of  the  Macintosh  screen.    Normally,  only  the  menu 
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House  Feature 

Walls 

Ridgeline 

Small  window 

Picture  window 

Door 

Door  window 

Door  knob 
Roof 

Chimney 

Smoke 


Erase  whole  screen 
Erase  wide  swaths 
Erase  fine  detail 


Macintosh  Feature(s)  Required 

Texture  and  line  thickness  selection. 
Filled  rectangle  or  paint  pot  and 
blank  rectangle. 

Line  thickness  selection  and  blank 
rectangle. 

Line  thickness  selection,  blank 
rectangle  and  "rubber-band"  lines. 

Line  thickness  selection  and  blank 
rectangle. 

Line  thickness  selection  and  blank 
rectangle. 

Line  thickness  selection  and  blank 
circle. 

Filled  circle. 

Texture  and  line  thickness  selection. 
Filled  "rubber-band"  lines  or  paint  pot 
and  blank  "rubber-band"  lines. 

Texture  and  line  thickness  selection. 
Filled  rectangle  or  paint  pot  and 
blank  rectangle. 

Pull-down  menu  for  brush  thickness, 
texture  selection  and  paintbrush. 

Backtracking  Commands 

Click  twice  on  eraser  icon. 

Select  and  use  eraser. 

Use  pull-down  edit  window  to  select 
Fatbits  option.    Use  move  screen 
icon  to  move  to  details  and  pencil 
icon  to  draw  or  erase. 


Figure  5:    Macintosh  Options  Required 
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name  is  there.  To  activate  them  the  subject  must  use  the  mouse  to  select  an  item  and 
display  its  window.  The  subject  must  then  systematically  explore  the  pull-down  win- 
dows until  he  finds  one  that  changes  the  thickness  of  the  paint  brush.  Most  did  not 
explore  this  thoroughly. 

There  is  no  obvious  connection  between  items  hidden  in  the  pull-down  menu  and 
the  selections  that  along  the  left  side  of  the  screen.  In  contrast  many  of  the  selections 
on  the  Macintosh,  brush  texture  is  difficult  to  find.  In  the  pull-down  menus,  the  Ma- 
cintosh is  more  like  a  conventional  computer,  where  options  are  hidden  or  only  obvi- 
ous to  the  expert. 

The  next  important  commands  for  this  experiment  are  backtracking  commands. 
In  order  to  correct  mistakes  a  subject  must  backtrack.  The  subjects  demonstrated 
considerable  willingness  to  use  the  backtracking  commands.  In  the  pretest,  the  sub- 
jects were  shown  how  to  do  a  couple  of  simple  types  of  backtracking.  There  are  three 
types  of  commands  on  Macpaint  that  allow  the  subjects  to  backtrack.  The  first  en- 
ables the  subject  to  erase  the  whole  screen.  This  is  done  by  clicking  twice  on  the  eraser 
icon. 

This  is  a  serious  backtracking  command.  It  destroys  everything  that  the  subject 
has  done  so  far.  The  second  backtracking  command  enables  the  subject  to  eraser  wide 
swaths  through  the  picture.  This  is  done  by  selecting  the  eraser  icon.  When  the  sub- 
ject selects  that,  a  blank  rectangle  about  a  quarter  of  an  inch  on  a  side  appears  on  the 
screen.  The  subject  manipulates  it  by  using  the  mouse.  Every  place  that  this  blank 
rectangle  touches  is  erased.  If  the  subject  has  a  steady  hand,  he  is  able  to  erase  some 
detail,  although  this  is  difficult. 

If  the  subject  wishes  to  erase  fine  detail,  then  a  more  sophisticated  method  is  re- 
quired. The  subject  must  use  a  pull-down  window  to  select  an  option  called  Fatbits. 
This  magnifies  the  screen  and  shows  a  pixel  as  a  rectangle  about  two  millimeters  on  a 
side.  The  subject  then  uses  the  pencil  icon  to  draw  or  erase  detail.  The  Fatbits  option 
is  difficult  to  find  and  no  subjects  used  it  intentionally. 

Figures  6-a  and  6-b  give  the  scoring  method  used  for  judging  the  house  diagram. 
The  house  diagram  was  judged  on  the  basis  of  topological  correctness.  Topological 
correctness  means  that  the  subject's  house  should  have  the  same  shape  as  the  original. 
It  also  had  to  have  the  same  textures  and  the  same  pieces  and  the  same  relationship. 
It  did  not  matter  if  the  subject's  house  was  the  same  size  as  the  original.  The  criterion 
for  judgement  was  shape,  texture  and  the  relationship  of  pieces. 

The  house  was  broken  up  into  features.  These  were  walls,  ridgeline,  small  window, 
picture  window,  door,  door  window,  door  knob,  roof,  chimney  and  smoke.  Each  of 
these  features  was  further  broken  down  into  the  elements  composing  them.  For  exam- 
ple, the  walls  were  broken  down  into  shape,  texture,  and  order.  The  shape  meant  that 
the  wall  had  to  be  rectangular.  Texture,  in  the  case  of  the  walls,  meant  that  the  subject 
had  to  select  the  brick  texture.    Border  meant  that  the  subject  had  to  recognize  that  the 
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House  Feature 

Walls 

Shape 

Texture 

Border 

Ridgeline 
Shape 
Texture 
Border 

Small  window 
Shape 
Texture 
Border 
Crosspieces 

Picture  window 
Shape 
Texture 
Border 

Door 

Shape 

Texture 

Border 

Door  window 
Shape 
Texture 
Border 

Door  knob 
Shape 
Texture 


Score 


Roof 


Shape 

Texture 

Border 


Figure  6-a:    Scoring  for  House  Drawing 
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Chimney 

Shape  1 

Texture  1 

Border  1 

Smoke 

Use  of  Pull-Down  Menu  1 

Texture  1 


Total  29 


Figure  6-b:    Scoring  for  House  Drawing 


goal  house  diagram  used  a  thicker  line  border  than  the  computer  was  producing  ini- 
tially.  The  subject  had  to  select  a  thicker  line. 

Each  of  these  elements  counted  a  score  of  one.  To  draw  the  house,  the  subject 
had  to  give  his  attention  to  each  of  these  items  in  sequence.  To  do  so  the  subject  had 
to  explore  the  computer  and  figure  out  how  to  shift  the  computer  status  from  one  item 
to  the  other.  In  the  process  of  exploration,  the  subject  had  to  be  able  to  backtrack.  In 
order  to  backtrack,  the  subject  had  to  use  the  backtracking  capabilities  outlined  in 
Figure  5. 

For  each  subject,  the  house  drawing  was  saved  to  a  disk  file.  Later,  the  drawings 
were  printed  and  used  to  compute  individual  scores  based  on  the  scoring  method  pre- 
sented in  Figures  6-a  and  6-b.  The  scores  for  individual  subjects  may  be  found  in  the 
Appendix.  There  was  one  exception  to  this.  One  of  the  subjects  was  something  of  a 
perfectionist  and  was  dissatisfied  with  the  house  produced.  So,  at  the  end  of  the  ex- 
perimental session,  she  clicked  the  erase  button  before  the  experimenter  could  save  the 
house. 

For  this  subject,  we  were  able  to  score  the  house  by  reviewing  the  video  recording. 
The  performance  of  this  subject  was  actually  quite  good.  The  subject  was  able  to  ex- 
plore the  computer,  recover  from  errors,  and  backtrack  effectively.  It  seemed  that  the 
subject's  performance  did  not  match  up  to  her  standards  of  excellence,  and  she  was 
frustrated. 

Figures  7-a  and  7-b  give  the  summary  statistics  for  the  subjects.  The  first  part 
summarizes  the  results  of  the  pretest.  The  detailed  pretest  and  the  instructions  the 
subjects  were  given  are  in  the  Appendix.  A  total  of  20  subjects  were  run.  Twelve  were 
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male  and  eight  were  female.  The  age  of  the  subjects  averaged  nearly  35  years  with  a 
standard  deviation  of  14.  The  oldest  subject  was  68  and  the  youngest  was  7. 

The  average  of  number  of  years  school  was  14.8  with  a  range  from  1  to  20.  In  this 
study,  we  were  not  trying  for  a  homogeneous  group  of  subjects.  Since  this  was  an  ini- 
tial exploratory  study,  we  used  a  heterogeneous  group.  After  we  know  more  about  ex- 
ploration, we  will  be  ready  for  homogeneous  groups  of  subjects.  Initial  studies  should 
use  heterogeneous  groups  to  isolate  characteristics  of  interest. 

Most  of  our  subjects  were  familiar  with  typewriters.  The  average  typing  speed  was 
a  respectable  33.3.  The  range  went  from  a  maximum  of  80  words  per  minute  down  to 
0.  The  question  about  typing  speed  was  included  only  to  establish  the  subjects  famil- 
iarity with  computer-like  equipment.  Typing  skill  was  not  required  in  our  experiment. 
All  interaction  is  done  via  the  Apple's  mouse.  The  keyboard  was  available  if  the  sub- 
jects wished  to  use  it.  For  the  purposes  of  the  house  drawing,  there  was  no  need.  The 
video  tape  showed  that  most  subjects  pushed  the  keyboard  out  of  the  way  during  the 
experiment. 

The  next  set  of  questions  is  our  attempt  to  elicit  information  about  advanced  or- 
ganizers. We  asked  subjects  if  they  liked  games,  and  65  percent  of  them  answered  po- 
sitively. The  games  reported  in  the  answers  (see  Appendix)  were  varied.  Subjects  re- 
ported board  games  and  card  games  most  frequently.  Surprisingly  enough,  video 
games  came  in  at  a  much  lower  percentage.  Only  40  percent  of  the  subjects  reported 
liking  or  playing  video  games.  These  reports  came  from  the  younger  subjects.  The 
older  subjects  were  familiar  with  video  games,  but  did  not  play  them. 

Fifty-five  percent  of  the  subjects  reported  themselves  as  comfortable  with  mathe- 
matics. One  should  be  careful  not  to  read  too  much  into  this.  This  is  a  self-reported 
level  of  feeling  about  mathematics  and  not  an  absolute  one.  One  of  the  subjects  who 
was  uncomfortable  with  mathematics  had  a  PH.D.  in  operations  research  from  MIT. 
Presumably  he  is  uncomfortable  with  mathematics  at  a  higher  level  than  most  of  the 
rest  of  us. 

Only  45  percent  of  the  subjects  reported  themselves  as  playing  a  musical  instru- 
ment. None  of  the  subjects  reported  any  high  degree  of  skill  of  music.  Most  of  those 
who  reported  playing  musical  instruments  were  referring  to  skills  learned  in  childhood 
and  long  since  abandoned. 

Seventy-five  percent  of  the  subjects  reported  that  they  knew  how  to  use  a  type- 
writer. Since  about  half  of  the  subjects  came  from  the  secretarial  pool  at  the  Naval 
Postgraduate  School,  this  is  not  surprising.  In  addition,  75  percent  of  the  subjects  re- 
ported they  knew  how  to  use  a  word  processor.  For  most  people  today,  use  of  a  word 
processor  and  use  of  a  typewriter  are  synonymous.   Typewriters  are  dying  out. 

Ninety  percent  of  the  subjects  reported  they  had  used  a  personal  computer.  Given 
the  high  level  of  penetration  of  personal  computers  in  the  schools  and  work  place,  this 
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Figure  7-a:    Summary  of  House  Drawing  Results 
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Figure  7-b:   Summary  of  House  Drawing  Results 


is  not  surprising.  Only  15  percent  of  the  subjects  reported  they  own  a  personal  com- 
puter. This  is  about  twice  the  national  average,  and  is  about  the  level  that  marketing 
surveys  expect  to  be  the  maximum  penetration  of  personal  computers  in  the  market- 
place. 

Ninety-five  percent  of  the  subjects  reported  they  had  used  a  computer.  This  in- 
cludes both  personal  computers  and  mainframe  computers.    This  means  that  one  of 
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our  subjects  was  a  mainframe  subject  while  all  of  the  others  had  used  either  personal 
computers  or  a  combination  of  mainframes  and  personal  computers.  None  of  our 
subjects  had  ever  used  a  Macintosh  before. 

This  is  somewhat  surprising  in  view  of  Apple's  massive  advertising  campaigns  since 
the  introduction  of  the  machine.  Virtually  all  of  our  subjects  were  aware  of  the  Ma- 
cintosh, but  apparently  this  awareness  did  not  extend  to  trying  to  find  an  opportunity 
to  use  the  machine.  So,  in  this  sense  we  started  with  a  group  of  naive  subjects.  None 
of  them  had  used  this  type  of  equipment  before. 

Since  this  was  an  exploratory  study,  we  had  no  clear  idea  of  what  we  might  find  as 
advance  organizers.  From  the  questions  we  asked  in  the  pretest  interview,  no  clear  in- 
dicators of  advance  organizers  emerged.  The  subjects  liked  games,  but  video  games 
were  not  strong  contenders.  The  games  that  they  did  mention  as  liking  did  not  seem  to 
be  likely  candidates  for  advance  organizers  in  the  task  assigned.  The  use  of  musical 
instruments  was  low  enough  in  the  group  that  it  does  not  seem  to  be  a  likely  candidate 
either. 

The  most  common  area  for  people  was  prior  use  of  computers  and  typewriters. 
Most  subjects  had  experience  with  these  machines,  and  during  the  test,  they  seemed  to 
regard  the  Macintosh  as  just  another  example.  The  interviews  and  review  of  the  video 
tape  did  not  show  any  particular  skill  that  the  subjects  were  falling  back  on  in  order  to 
do  this  task.  Based  on  what  we  observed,  the  search  for  advance  organizers  in  this 
task  would  have  to  be  deemed  a  failure. 

The  subjects  performance  in  house  drawing,  however,  was  anything  but  a  failure. 
The  average  house  score  was  15  with  a  standard  deviation  of  7.3.  The  subjects  scores 
range  from  a  minimum  of  zero  to  a  maximum  of  27.  Most  subjects  performed  quite 
creditably  on  the  task. 

When  we  reviewed  the  performance  on  the  individual  subtasks,  we  see  the  subjects 
largely  confined  themselves  to  the  major  tasks  involved  in  drawing  the  house.  Most 
subjects  were  able  produce  the  major  features,  the  walls,  the  small  window,  picture 
window,  the  door,  the  roof,  and  the  chimney.  When  we  come  to  the  less  important 
features  of  the  house,  we  find  less  compliance  with  the  model.  Only  40  percent  of  the 
subjects  got  the  ridge  line,  10  percent  were  able  to  reproduce  the  door  knob,  and  only 
10  percent  were  able  to  find  the  pull-down  menu  to  create  smoke  correctly. 

The  subjects  were  shown  how  to  use  a  pull-down  menu  in  pretest  orientation,  but 
most  of  them  either  forgot  or  decided  that  this  was  not  crucial  to  the  test.  In  our  re- 
views of  video  tapes,  we  have  only  a  minority  of  the  subjects  using  the  pull-down  men- 
us. Even  those  who  use  them  do  not  explore  them  fully.  Most  of  the  shapes  involved 
in  the  house  were  rectilinear  or  polygonal.  The  subjects  did  well  on  these,  and  had  no 
problems  in  shifting  from  one  shape  to  another. 
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When  the  shapes  were  curved,  as  with  the  door  knob  and  the  door  window,  the 
subjects  had  more  difficulty.  In  many  cases,  they  missed  the  proper  icons.  In  the  case 
of  the  door  knob,  many  subjects  did  not  use  the  filled  circle.  They  tried  drawing  the 
door  knob  free,  hand  using  a  variety  of  other  icons.  When  the  results  were  not  what 
they  had  hoped  for,  most  of  the  subjects  moved  on,  apparently  considering  the  door 
knob  too  trivial  to  bother  with.  They  already  had  substantial  investment  in  drawing 
the  door,  and  erasing  the  door  knob  would  have  destroyed  that.  This  happened  in  vir- 
tually all  of  the  subjects.  Where  the  investment  was  viewed  as  substantial,  there  was 
no  desire  to  backtrack. 

Something  of  the  same  sort  seems  to  have  occurred  in  drawing  the  smoke.  Ap- 
parently the  smoke  was  seen  as  amorphous  enough  that  there  was  no  need  to  work 
hard  in  producing  results  like  the  model.  This  as  well  as  the  difficulty  in  finding  pull- 
down menus  may  have  been  a  reason  for  the  low  score  for  smoke.  Subjects  explored 
the  pull-down  menus  but  they  did  not  try  hard  to  duplicate  the  quality  of  smoke  in  the 
model  house.  Also,  the  smoke  was  the  last  task.  They  were  nearing  the  end  of  the  ex- 
periment and  most  were  ready  to  move  on. 

In  summary,  the  house  task  worked  well  in  eliciting  exploratory  behavior  from  the 
subjects.  In  future  experiments,  I  would  recommend  putting  in  more  details  to  force 
more  exploration.  Details  should  be  added  that  would  force  the  subjects  to  transition 
from  one  icon  to  another  in  performing  the  task.  This  experiment  did  reasonably  well 
at  that. 

Figure  8  presents  Carroll's  list  of  problems  typically  faced  by  subjects  in  an  ex- 
ploratory environment.  It  is  interesting  to  compare  this  list  with  the  kinds  of  problems 
faced  by  our  subjects  on  the  Macintosh.  The  Macintosh  is  a  good  test  of  exploratory 
environments.    It  is  not  as  intimidating  as  other  computers. 

Carroll  lists  the  characteristics  disorientation,  emptiness,  and  paradox.  Disorien- 
tation means  that  the  player  does  not  know  what  to  do  in  the  environment.  Emptiness 
means  the  computer  offers  few  hints  to  help  achieve  goals.  This  was  definitely  not  the 
case  in  the  Macintosh.  Disorientation  was  kept  to  a  minimum  by  hints  for  action 
around  the  screen.  Emptiness  was  not  a  Macintosh  characteristic.  Paradox  was  sel- 
dom a  problem  either.  The  Macintosh  rarely  told  the  player  any  course  of  action  that 
was  inappropriate. 

Some  of  the  other  problems  faced  by  subjects  of  exploratory  environments  were 
present.  The  illusiveness,  mystery  messages,  slipperyness  and  side  effects  could  also  be 
a  problem  on  the  Macintosh.  In  observing  the  tapes  of  the  subjects,  we  noticed  that 
several  of  them  managed  to  activate  a  menu  feature  called  "Fatbits".  There  are  two 
ways  to  do  this.  One  way  involves  choosing  that  selection  from  a  pull  down  menu. 
One  subject  did  this  on  purpose.  The  subject  explored  the  feature,  decided  it  was  not 
useful,  and  went  on. 
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Disorientation 
Illusiveness 
Emptiness 
Mystery  Messages 
Slipperiness 

Side  effects 

Paradox 
Laissez-faire 


The  user/player  doesn't  know  what  to  do  in  the 
system  environment. 

What  the  user/player  wants  to  do  is  deflected 
toward  other,  perhaps  undesired,  goals. 

The  screen  is  effectively  vacant  of  hints  as 
to  what  to  do  or  what  went  wrong. 

The  system  provides  feedback  that  is  useless 
and/or  misleading. 

Doing  the  "same  thing"  in  different 
situations  has  unexpectedly  different 
consequences. 

Taking  an  action  has  consequences  that  are 
unintended  and  invisible  but  cause  trouble 
later. 

The  system  tells  the  learner/player  to  do 
something  that  is  clearly  inappropriate. 

The  system  provides  no  support  or  guidance 
for  overall  goals  (e.g.  "winning" 
"typing  a  letter") 


Figure  8:   Typical  Problems  Faced  by  Users  -  Carrollf  1982a] 


The  other  way  to  activate  it  is  to  click  twice  on  the  pencil  icon.  It  is  easy  to  do 
this  latter  approach  inadvertently.  If  the  subject  does  this,  the  Macintosh  zooms  in  on 
a  portion  of  the  picture  and  expands  it  to  large  size.  The  individual  pixels  that  make 
up  the  screen  are  expanded  to  about  2  millimeters  on  a  side.  The  user  is  able  to  turn 
on  and  off  individual  pixels.  The  picture  on  the  screen  bears  some  resemblance  to  the 
original  picture,  but  it  is  definitely  not  the  same.  Nor  is  it  entirely  clear  how  the  user 
got  into  this  mode. 

Every  user  of  MacPaint  has  at  one  time  or  another  activated  this  feature.  The  first 
time  it  happens,  it  is  quite  confusing.  This  was  the  case  with  many  of  our  subjects. 
They  got  into  the  Fatbits  option  inavertently  and  the  results  were  confusing  for  them. 
Interestingly  enough,  all  were  able  to  recover  and  go  on  to  draw  completely  acceptable 
houses.  The  subjects  who  got  into  this  option  were  the  ones  who  explored  successfully. 
The  less  successful  explorers  didn't  get  into  this  kind  of  confusion.    Whether  they 
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would  have  been  able  to  get  out  of  it  is  an  open  question.  In  general,  the  Macintosh 
provided  a  reasonable  and  challenging  exploratory  environment. 

The  next  set  of  comments  are  general  observations  on  the  ways  in  which  subjects 
explored.  The  sample  that  we  used  was  not  large  enough  to  break  down  into  specific 
sets  of  statistically  valid  observations.  It  was  intentionally  chosen  as  a  diverse  sample 
to  see  what  kinds  of  exploration  we  could  elicit.  In  future  studies,  it  would  be  wise  to 
choose  homogeneous  groups  from  the  target  population  of  interest.  The  results  of  the 
experiments  presented  here  could  offer  some  guidance  in  what  to  look  for  and  how  to 
choose  these  groups. 

Several  things  were  obvious  in  reviewing  the  exploration  tapes  dealing  with  older 
subjects.  In  the  first  place,  these  subjects  did  not  tend  to  explore  the  options  available. 
They  seemed  to  be  far  more  task  oriented  and  this  seemed  to  inhibit  exploration.  Their 
main  function  seemed  to  be  focusing  on  getting  the  job  done.  Exploration  was  regard- 
ed as  something  that  inhibited  performance  of  the  task.  They  would  find  a  few  satis- 
factory icons  and  then  use  these  in  brute  force  mode  to  get  the  job  done.  The  idea 
seemed  to  be  "stick  with  what  you  know  and  show  that  you  are  working". 

This  is  an  attitude  that  is  frequently  seen  among  users  of  computer  equipment.  It 
is  a  self  defeating  attitude  because  usage  of  computer  equipment  involves  a  high  degree 
of  learning.  Perhaps  25%  of  the  job  should  be  spent  in  learning  or  exploration.  Such 
subjects  are  likely  to  be  trainable  in  a  specific  narrow  area  of  computing.  They  are 
likely  to  have  trouble  broadening  out  and  expanding  their  horizons  to  include  new 
technologies. 

The  different  problems  surfaced  when  we  were  dealing  with  high  status  subjects. 
Such  subjects  seemed  to  have  difficulty  in  taking  feedback  from  the  machine.  Since 
these  tended  to  be  older  subjects  as  well,  this  did  compound  some  of  the  observations. 
Such  subjects  are  used  to  giving  vague  commands  to  secretaries.  Secretaries  are  then 
responsible  for  filling  in  the  details  of  the  command  and  producing  the  desired  result. 
The  subjects  themselves  are  not  responsible  for  the  details. 

We  ran  into  one  subject  who  needed  approval  from  the  experimenter.  This  subject 
had  to  ask  before  any  selection  was  made  whether  she  was  doing  it  right.  She  seemed 
to  be  quite  disturbed  with  the  idea  that  exploration  was  all  right.  This  subject  was  a 
young  girl,  and  had  probably  been  socialized  into  the  idea  that  she  should  be  "good". 
This  meant  that  just  being  turned  loose  on  an  expensive  computer  with  only  vague  in- 
structions about  what  to  do  was  not  entirely  acceptable. 

It  was  interesting  to  contrast  the  attitude  of  this  subject  with  the  attitude  of  a 
young  boy,  about  one  year  older.  The  boy's  performance  on  the  machine  was  not 
substantially  different  from  the  young  girl's.  The  attitudes,  however,  were  miles  apart. 
They  young  boy  took  the  attitude  that  he  knew  what  he  was  doing  and  how  to  do  it 
and  needed  absolutely  no  help  or  instruction  from  the  experimenter.  Our  timid  subject 
was  the  only  such  subject  we  observed  in  this  experimental  group,  although  any  teach- 
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er  of  computer  science  has  seen  similar  subjects  in  the  student  population.  Such  indi- 
viduals rarely  become  good  at  computing  and  are  poor  candidates  for  any  kind  of  work 
in  technological  literacy. 

Another  subject  that  we  observed  is  what  I  would  classify  as  a  conservative  non- 
user.  This  person  is  not  a  regular  user  of  computing  equipment  but  needs  to  use  it  oc- 
casionally for  employment  reasons.  Doing  so,  this  individual  has  developed  a  conser- 
vative strategy.  This  strategy  includes  taking  small  exploratory  steps,  saving 
intermediate  output,  and  proceeding  from  one  stable  state  to  another.  The  subject  was 
willing  to  indulge  in  limited  backtracking,  but  only  enough  to  perform  the  task  at  hand. 
This  subject's  strategies  were  quite  effective  and  worked  well  for  the  individual.  It  is 
unlikely  that  this  individual  will  become  a  true  computer  "power  user"  but  the  strategy 
works  well  for  somebody  who  has  limited  need  of  computing  power. 

The  secretarial  staff  constituted  the  most  effective  group  of  subjects  on  the  Ma- 
cintosh. Their  educational  level  was  below  that  of  the  faculty  and  computer  center 
subjects,  but  they  were  able  to  use  the  machine  extremely  effectively.  They  were  used 
to  using  word  processing  equipment,  although  this  equipment  was  very  different  from 
the  Macintosh.  There  is  nothing  in  their  ordinary  jobs  skills  that  related  directly  to  the 
tests  they  were  asked  to  do.  The  most  notable  feature  of  the  secretarial  subjects  was 
their  attention  to  detail  and  their  willingness  to  take  feedback  from  the  machine.  These 
are  tasks  that  are  required  in  ordinary  word  processing,  and  there  appeared  to  be  some 
transfer  in  the  way  they  used  the  Macintosh.  These  are  skills  that  should  probably  be 
included  in  any  attempt  to  teach  technological  literacy.  Whether  these  skills  can  be 
successfully  transfered  to  high  status  subjects  is  another  question  for  another  research 
project. 


Conclusions  and  Areas  for  Future  Research 


In  our  subjects,  it  seemed  that  it  would  be  a  mistake  to  look  for  inherent  skills  or 
intelligence  levels  as  a  factor  in  performing  the  assigned  tasks.  All  subjects  were  intel- 
ligent enough  to  be  able  to  handle  it.  The  difference  is  in  performance  level  seem  to  be 
related  to  several  things,  motivation,  attitude,  and  attention.  Motivation  seemed  to  be 
a  primary  factor  in  the  performance  of  many  subjects.  Those  who  took  a  nonchalant 
attitude  towards  the  task  did  not  explore  much  and  as  a  consequence  performed  more 
poorly  on  the  assigned  task. 

Attitude  about  the  task  also  was  important.  Those  who  took  a  narrowly  focused 
attitude  towards  the  task  did  not  indulge  in  much  exploration.  As  a  consequence  they 
did  not  discover  the  features  they  needed  to  fully  complete  the  assigned  task.  The  final 
factor  was  the  attention  to  detail.  If  the  subject  was  not  used  to  paying  attention  to 
detail  or  not  used  to  paying  attention  to  feedback  on  the  screen,  the  performance  in 
the  task  was  not  as  good. 

The  issue  is  the  extent  to  which  this  experiment  was  relevant  to  the  larger  areas  of 
technological  literacy.  This  is  one  that  still  unanswered.  The  range  of  tasks  under  the 
heading  of  technological  literacy  is  so  broad  that  the  same  objection  could  be  raised  to 
any  experiment.  Does  this  task  embody  the  features  that  one  would  expect  to  find  in  a 
programming  task,  a  mechanical  diagnosis  task,  or  the  operation  of  some  complex 
piece  of  equipment?  My  intuitive  feeling  after  watching  people  perform  on  this  is  that 
it  probably  does  but  there  is  no  way  to  prove  this.  This  experiment  was  largely  ex- 
ploratory, although  it  does  have  implications  for  both  the  teaching  of  technological  lit- 
eracy and  for  future  research  in  this  area. 

We  feel  strongly  that  exploration  is  a  suitable  paradigm  for  beginning  the  study  of 
technological  literacy.  Carroll's  taxonomy  of  exploratory  environment  characteristics 
given  in  Figure  2  is  a  reasonable  place  to  start.  To  some  degree  or  another,  this  para- 
digm characterises  all  exploratory  environments  of  interest  in  technological  literacy. 
All  such  environments  have  a  certain  responsiveness.  When  you  do  something  you  get 
some  feedback.  Feedback  may  be  a  change  on  the  screen,  a  change  in  the  performance 
of  an  engine,  or  some  other  system  characteristic.  We  observed  that  it  is  necessary  to 
tell  the  users  how  to  pay  attention  to  this  feedback.  For  many  of  our  subjects,  atten- 
tion to  feedback  was  a  problem. 

Related  to  responsiveness  is  the  idea  of  benchmarks.  A  subject  needs  to  know 
where  he  is  in  a  given  episode  or  session.  A  course  on  technological  literacy  would 
also  include  how  to  pay  attention  to  these  benchmarks. 
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Acceptable  uncertainty  and  safe  conduct  are  two  characteristics  of  an  exploratory 
environment.  The  subject  needs  to  know  that  it  is  all  right  to  be  less  than  fully  confi- 
dent of  your  understanding.  The  courses  that  subjects  have  had  in  technical  areas  lead 
them  to  believe  there  is  one  right  answer  for  everything.  Mathematics  is  a  common 
problem  in  this  respect.  Students  are  always  feeling  like  nothing  they  ever  do  is  right. 
When  they  carry  this  attitude  over  to  an  exploratory  environment,  it  very  much  inhib- 
its their  ability  to  work  effectively  in  that  environment. 

Acceptable  uncertainty  and  safe  conduct  emphasize  that  there  is  nothing  that  you 
can  do  wrong  in  this  environment  that  cannot  be  fixed.  A  course  in  technological  lit- 
eracy would  want  to  emphasize  these  characteristics  of  an  exploratory  environment. 

The  last  four  elements  in  the  paradigm  emphasize  the  way  in  which  a  subject 
learns  in  an  exploratory  environment.  These  include  learning  by  doing,  opportunity, 
taking  charge  and  control.  The  subject  needs  to  know  that  learning  is  an  active  and 
not  a  passive  process.  Much  of  teaching  that  people  are  subjected  to  is  of  the  passive 
sort.   You  go,  you  listen  to  somebody  talk,  you  take  a  test  and  that  is  it. 

This  does  not  work  in  an  exploratory  environment  and  is  not  an  acceptable  ap- 
proach to  acquiring  technological  literacy.  Any  course  that  claims  to  teach  somebody 
technological  literacy  must  emphasize  these  features  of  the  environment.  Even  more 
so,  it  must  emphasize  that  it  is  acceptable  for  subjects  to  proceed  in  this  way. 

One  problem  we  observed  was  that  some  subjects  did  not  feel  it  was  acceptable  to 
learn  in  a  working  environment.  Learning  by  doing  was  foreign  to  them.  The  only 
acceptable  thing  in  a  work  environment  was  to  put  your  blinders  on  and  charge.  Such 
subjects  did  not  seem  to  be  able  to  take  charge.  They  did  not  seem  to  have  the  feeling 
that  they  were  in  control  of  the  process. 

Our  initial  goal  of  trying  to  find  a  mental  image  that  contributed  to  the  subject's 
ability  to  solve  problems  was  a  failure.  There  was  nothing  that  we  could  see  in  the 
mental  images  that  the  subjects  had  that  had  any  effect  on  performance.  Far  more 
important  than  mental  image  was  the  subject's  attitudes  in  the  experiment. 

Our  subjects  seemed  to  be  setting  goals  implicitly.  Unfortunately,  we  did  not  de- 
velop a  measure  for  the  way  that  these  goals  affected  their  performance  in  the  task. 
Some  of  our  subjects  appeared  to  have  highly  task  oriented  goals  and  these  led  away 
from  any  acceptance  of  explorations.  The  subject's  attitude  seemed  to  be  far  more  im- 
portant than  any  kind  of  mental  image.  A  measure  of  attitude,  especially  attitude  re- 
lated to  work,  would  probably  be  far  more  useful  as  a  way  of  predicting  task  perform- 
ance. 

Another  feature  that  should  be  looked  at  is  the  affective  component  that  the  sub- 
ject brings  to  the  experiment.  In  designing  a  course  for  technological  literacy,  rein- 
forcement of  the  positive  affect  may  be  very  useful.  Subjects  have  had  many,  many 
years  of  reinforcement  of  looking  for  the  right  answer  in  a  course.  There  is  no  right 
answer  in  exploration.   There  arc  only  effective  or  ineffective  ways  of  reaching  a  goal. 
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Perhaps  it  would  be  useful  for  a  course  in  technological  literacy  to  bolster  the  con- 
fidence of  the  students.  It  should  try  to  undo  the  damage  done  by  many  years  of  con- 
ventional teaching.  It  is  probably  a  mistake  in  designing  courses  for  people  to  look  for 
aptitude.  It  would  seem  based  on  these  preliminary  results  that  attitude  is  far  more 
important. 

If  I  were  designing  a  course  for  technological  literacy,  these  are  the  points  I  would 
consider: 

1.  Teaching  general  problem  solving  skills  is  likely  to  be  more  useful  than  spe- 
cific skills.  But  these  general  problem  solving  skills  should  be  taught  in  the 
context  of  a  course,  and  they  should  be  course  specific. 

2.  A  course  should  reward  for  exploration  rather  than  for  the  right  answer. 

3.  The  course  should  teach  students  to  pay  attention  to  the  feedback  that  they 
get  from  the  environment. 

4.  The  course  should  bolster  confidence  in  the  students  and  teach  them  that  it 
is  all  right  to  make  mistakes.  It  should  emphasize  the  forgiving  nature  of  an 
exploratory  environment. 

5.  The  course  should  emphasize  that  the  learning  process  is  something  that 
takes  place  between  the  student  and  the  machine.  It  is  not  something  that 
takes  place  between  the  student  and  the  instructor.  This  is  very  much  a 
process  of  learning  by  doing. 

6.  The  course  should  bolster  the  idea  of  the  generalizablity  of  the  material.  It 
should  emphasize  that  the  material  presented  can  be  used  in  a  variety  of 
areas. 

Future  research  in  this  area  should  look  at  a  variety  of  points.  These  are:  First,  it 
should  develop  a  taxonomy  of  subjects  of  various  groups  of  interest.  Such  groups 
would  include  grouping  by  age,  sex,  and  prior  background.  Two,  within  these  various 
interest  groups,  try  to  develop  certain  taxonomies  of  problem  solving  styles.  These 
could  include  willingness  to  explore,  different  approaches  to  exploration.  Three,  once 
these  have  been  developed,  testing  could  begin  on  a  variety  of  experimental  treatments 
for  teaching  exploration  or  technological  literacy  skills. 


Appendix  A 
Instructions  to  the  Subject 

Subject  Instructions 

This  is  a  Macintosh  Computer.  The  computer  consists  of  the  screen,  to  display 
the  computer's  response  to  your  commands,  and  a  keyboard  and  a  "mouse",  which  are 
instruments  used  to  command  the  computer  to  do  what  you  wish.  In  the  present  ex- 
periment, you  will  be  using  only  the  mouse  to  communicate  to  the  computer  what  you 
want  it  to  do.  The  mouse  is  used  to  position  the  cursor  on  the  computer  screen.  To 
issue  your  command  to  the  computer  simply  push  the  button  located  on  top  of  the 
mouse.  In  turn,  the  computer  will  respond  accordingly  by  displaying  what  you  com- 
manded it  to. 

This  experiment  should  be  fun.  Look  at  it  as  an  adventure  in  exploring  the  world 
of  the  computer.  All  you  are  required  to  do  is  to  draw  the  picture  of  the  house  (show 
subject  the  house).  You  must  try  to  draw  the  house  using  the  computer  exactly  the 
way  it  appears  in  the  picture.  I  will  not  provide  any  hints  as  to  how  the  picture  can  be 
drawn.  You  will  not  be  judged,  there  are  no  right  or  wrong  answers.  The  purpose  of 
this  experiment  is  merely  to  observe  how  people  explore  new  computer  environments. 
You  will  have  45  minutes  to  complete  this  picture. 

Let  me  explain  in  more  detail  how  the  mouse  works.  Simply  move  the  mouse 
around  on  the  table  beside  your  computer  screen  and  you  will  see  the  cursor  move  on 
the  screen.  Push  the  button  on  the  mouse  once  over  the  picture  showing  the  function 
you  wish  to  select  and  push  the  button  twice  to  execute  the  selected  function  (demon- 
strate the  movement  of  the  cursor  with  the  mouse  on  the  screen). 

In  case  you  discover  you  have  made  an  error  and  you  want  to  erase  a  portion  of 
what  you  have  drawn,  simply  position  the  cursor  with  the  mouse  over  the  picture  of 
the  eraser  on  the  menu  and  push  the  button  on  the  mouse  once  to  select  the  function. 
Then  move  the  cursor  over  the  mistake  in  your  drawing  and  move  the  cursor  back  and 
forth  as  if  you  were  erasing  a  chalkboard.  To  erase  the  entire  picture,  position  the 
cursor  over  the  picture  of  the  eraser  and  push  the  button  on  the  mouse  twice. 

Are  there  any  questions? 

Note  -  the  experimenter  demonstrates  the  use  of  the  mouse  to  select  an  icon,  draw  a 
line,  access  a  pull-down  menu  and  erase  a  whole  picture. 
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Appendix  B 
COMPUTER  EXPLORATION  PRETEST 


Name:  Date: 


1.  How  old  are  you? 

2.  How  many  years  of  schooling  have  you  had? 

3.  Do  you  like  to  play  games? 

4.  What  games  do  you  like  to  play? 

a)  Chess 

b)  Bridge 

c)  Go 

d)  Poker 

5.  Do  you  like  video  games? 

a)  Space  Invaders 

b)  PacMan 

c)  Asteroids 

d)  Adventure  type  games 

e)  Other 

6.  Are  you  comfortable  with  mathematics? 

7.  What  mathematics  courses  have  you  taken? 
a)     Arithmetic  (years?) 
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b)  Algebra 

c)  Geometry 

d)  Trigonometry 

e)  Calculus 

f)  Logic 

g)  Other 

8.  Do  you  play  a  musical  instrument? 

a)  Instrument 

b)  Years 

9.  Do  you  read  music? 

10.  Do  you  type? 
a)     How  fast? 

11.  Have  you  ever  used  a  word  processor? 

12.  Do  you  own  a  personal  computer? 

13.  Have  you  ever  used  a  computer? 

14.  What  computer  languages  have  you  ever  used? 

a)  BASIC 

b)  PASCAL 

c)  FORTRAN 

d)  LOGO 

e)  Other 

15.  Have  you  ever  used  a  Macintosh? 


Appendix  C 
Posttest  Interview 

Is  this  computer  like  anything  you  have  used  before? 
Typewriter? 
Pencil  and  paper? 
Paintbrush? 
Drafting  tools? 
Other  computers  and  wordprocessors? 

What  would  you  use  this  computer  for? 
What  could  make  this  computer  easier  to  use? 
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Appendix  D 

Subject  Notes 

Subject  Number:    1 

Sex:    F       Age:   43         Years  of  Sch 

doI:    20 

Likes  games: 

1 

Types: 

Video  Games: 

0 

Comfortable  with  Math: 

1 

Math  Courses: 

Arithmetic,  Algebra,  Geometry,  Trig, 

Calculus,  Logic,  Statistics 

Musical  Instrument: 

1 

Typing  Skills 

Typewriter: 

1 

Speed: 

70 

Word  Processor: 

1 

Used  a  PC: 

1 

Used  a  computer: 

1 

Languages: 

BASIC,  PASCAL,  FORTRAN,  COB 

Used  a  Macintosh: 

0 

House  Score: 

15 

House  Feature 


Score 


Walls 


Shape 

Texture 

Border 


Ridgeline 
Shape 
Texture 
Border 

Small  window 
Shape 
Texture 
Border 
Crosspieces 
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Picture  window 
Shape 
Texture 
Border 

Door 

Shape  1 

Texture  1 

Border 

Door  window 

Shape  1 

Texture  1 

Border  1 

Door  knob 
Shape 
Texture 

Roof 

Shape 

Texture  1 

Border 

Chimney 

Shape  1 

Texture  1 

Border 

Smoke 

Use  of  Pull-Down  Menu 

Texture  1 


Total  15 


Notes  -  The  subject  felt  that  the  primary  goal  was  to  reproduce  the  house.  The  subject 
did  not  really  feel  free  to  explore.  Work  drove  the  experiment.  The  subject  used  a 
brute  force  approach  and  tried  to  recreate  icon  functions  manually.  This  took  most  of 
the  allotted  time  for  the  experiment.  The  subject  began  exploration  toward  the  end  of 
the  experiment. 
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Subject  Number:   2 


Sex:  M         Age: 

33          Years  of  School: 

19 

Likes  games: 

0 

Types: 
Video  Games: 

0 

Comfortable  with  Math: 

1 

Math  Courses: 

Arithmct 

ic,  Algebra,  Geometry,  Trig, 

Calculus 

Logic,  Statistics 

] 

Musical  Instrument: 

1 

Typing  Skills 

Typewriter: 

Speed: 

Word  Processor: 

1 
20 

1 

Own  a  PC: 

1 

Used  a  PC: 

1 

Used  a  computer: 
Languages: 

1 
BASIC, 

PASCAL,  FORTRAN,  LISF 

Used  a  Macintosh: 

0 

House  Score: 

25 

House  Feature 

Score 

Walls 

Shape 
Texture 

1 
1 

Border 

1 

Ridgeline 

Shape 
Texture 

1 
1 

Border 

1 

Small  window 

Shape 
Texture 

1 
1 

Border 

1 

Crosspieces 

1 

Picture  window 

Shape 
Texture 

1 

Border 

1 

Door 

Shape 
Texture 

1 
1 

44 


Border  1 

Door  window 

Shape       .  1 

Texture 

Border  1 

Door  knob 
Shape 
Texture  1 

Roof 

Shape  1 

Texture  1 

Border  1 

Chimney 

Shape  1 

Texture  1 

Border 

Smoke 

Use  of  Pull-Down  Menu  1 

Texture  1 


Total  25 
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Subject  Number:    3 


Sex:    M      Age: 

38      Years  of  School: 

18 

Likes  games: 

1 

Types: 

1 

Video  Games: 

1 

Comfortable  with  Math: 

1 

Math  Courses: 

Arithmetic,  Algebra,  Geometry,  Trig, 
Calculus,  Logic,  Statistics 

Musical  Instrument: 

1 

Typing  Skills 

Typewriter: 

1 

Speed: 

30 

Word  Processor: 

1 

Own  a  PC: 

0 

Used  a  PC: 

1 

Used  a  computer: 

1 

Languages: 

BASIC,  PASCAL,  FORTRAN,  SPSS,  SAS 

Used  a  Macintosh: 

0 

House  Score: 

28 

House  Feature 

Score 

Walls 

Shape 

Texture 

Border 

Ridgeline 

Shape 

Texture 

Border 

Small  window 

Shape 

Texture 

Border 

Crosspieces 

Picture  window 

Shape 

Texture 

Border 

Door 

Shape 

Texture 
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Border  1 

Door  window 

Shape  1 

Texture  1 

Border  1 

Door  knob 
Shape 

Texture  1 

Roof 

Shape  1 

Texture  1 

Border  1 

Chimney 

Shape  1 

Texture  1 

Border  1 

Smoke 

Use  of  Pull- Down  Menu  1 

Texture  1 


Total  28 
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Subject  Number:   4 


Sex:  F        Age: 

32        Years  of  School: 

13 

Likes  games: 

1 

Types: 
Video  Games: 

Chess 

1 

,  Poker 

Comfortable  with  Math: 

0 

Math  Courses: 

Arithmetic 

Musical  Instrument: 

0 

Typing  Skills 

Typewriter: 

Speed: 

Word  Processor: 

1 

75 
1 

Own  a  PC: 

0 

Used  a  PC: 

1 

Used  a  computer: 
Languages: 
Used  a  Macintosh: 

1 
0 

House  Score: 

22 

House  Feature 

Score 

Walls 

Shape 
Texture 

1 
1 

Border 

1 

Ridgeline 

Shape 
Texture 

Border 

Small  window 

Shape 
Texture 

1 
1 

Border 

Crosspieces 

1 

Picture  window 

Shape 
Texture 

1 
1 

Border 

1 

Door 


Shape 
Texture 
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Border  1 

Door  window 

Shape  1 

Texture  1 

Border  1 

Door  knob 
Shape 
Texture  1 

Roof 

Shape  1 

Texture  1 

Border  1 

Chimney 

Shape  1 

Texture  1 

Border  1 

Smoke 

Use  of  Pull-Down  iMenu 
Texture 


Total  22 


Notes  -  Subject  initially  used  only  the  demonstrated  functions  and  seemed  reluctant  to 
try  others.  The  subject  discovered  the  most  versatile  icon  (rubber-band  lines)  early  and 
used  it  to  draw  the  whole  house. 
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Subject  Number:   5 


:    M      Age 

37       Years  of  School: 

20 

Likes  games: 

1 

Types: 
Video  Games: 

Poker 
0 

Comfortable  with  Math: 

1 

Math  Courses: 

Arithmetic,  Algebra,  Geometry, 
Calculus,  Logic,  Statistics 

Ti 

Musical  Instrument: 

0 

)ing  Skills 

Typewriter: 

Speed: 

Word  Processor: 

o 

0 

Own  a  PC: 

Used  a  PC: 

Used  a  computer: 
Languages: 

BASIC,  PASCAL,  FORTRAN, 

C 

Used  a  Macintosh: 

0 

House  Score: 

IS 

I  rig, 


House  Feature 


Score 


Walls 


Shape 

Texture 

Border 


Ridgeline 
Shape 
Texture 
Border 


Small  window 
Shape 
Texture 
Border 
Crosspieces 

Picture  window 
Shape 
Texture 
Border 


Door 


Shape 
Texture 


50 


Border  1 

Door  window 
Shape 
Texture 
Border 

Door  knob 
Shape 
Texture 

Roof 

Shape  1 

Texture 

Border  1 

Chimney 

Shape  1 

Texture  1 

Border  1 

Smoke 

Use  of  Pull-Down  Menu 
Texture 


Total  18 
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Subject  Number:   6 
Sex:  F        Age:  29 


Years  of  School:  12 

Likes  games:  1 

Types:  Chess,  Poker 
Video  Games:  1 

Comfortable  with  Math:  1 

Math  Courses:  Arithmetic,  Algebra,  Geometry 


Musical  Instrument: 


o 


Typing  Skills 

Typewriter: 

Speed: 

Word  Processor: 

1 
50 

1 

Own  a  PC: 

0 

Used  a  PC: 

1 

Used  a  computer: 
Languages: 

I 

BASIC,  COBOL 

Used  a  Macintosh: 

0 

House  Score: 

12 

House  Feature 

Score 

Walls 

Shape 
Texture 

1 
1 

Border 

1 

Ridgeline 

Shape 
Texture 

1 

Border 

Small  window 

Shape 
Texture 

1 

1 

Border 

1 

Crosspieces 

Picture  window 

Shape 
Texture 

Border 

Door 

Shape 
Texture 
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Border 

Door  window 
Shape 
Texture 
Border 

Door  knob 
Shape 
Texture 

Roof 

Shape 

Texture  1 

Border  1 

Chimney 

Shape  1 

Texture  1 

Border  1 

Smoke 

Use  of  Pull-Down  Menu 
Texture 


Total  12 


Notes  -  Subject  experimented  with  the  demo  functions  first  and  decided  to  fill  in  back- 
ground texture  first  and  then  erase  into  a  rectangular  shape.  Conservative  -  selected  a 
strategy  early  and  stuck  with  it. 
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Subject  Number:    7 


Sex:  M         Age:  8          Years  of  Sch 

ool: 

2 

Likes  games 

1 

Types 

Video  Games. 

1 

Comfortable  with  Math 

1 

Math  Courses 

Arithmetic,  Algebra,  Geometry,  Trig, 
Calculus,  Logic,  Statistics 

Musical  Instrument 

1 

Typing  Skills 

Typewriter 

0 

Speec 

1: 

Word  Processor 

0 

Own  a  PC 

0 

Used  a  PC 

1 

Used  a  computer 

1 

Language; 

Used  a  Macintosh 

0 

House  Score 

13 

House  Feature 

Walls 

Shape 

Texture 

Border 

Ridgeline 
Shape 
Texture 
Border 

Small  window 
Shape 
Texture 
Border 
Crosspieces 

Picture  window 
Shape 
Texture 
Border 


Score 
1 


Door 


Shape 
Texture 
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Border 

Door  window 

Shape       .  1 

Texture  1 

Border 

Door  knob 

Shape  1 

Texture 

Roof 

Shape  1 

Texture 

Border 

Chimney 

Shape  1 

Texture 

Border 

Smoke 

Use  of  Pull-Down  Menu 
Texture 


Total  13 
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Subject  Number:    8 


Sex:  F        Age:  24          Years  of  School: 

13 

Likes  games: 

1 

Types: 

Chess 

Video  Games: 

1 

Comfortable  with  Vlath: 

0 

Math  Courses: 

Arithmetic,  Algebra 

Musical  Instrument: 

1 

Typing  Skills 

Typewriter: 

1 

Speed: 

M) 

Word  Processor: 

1 

Own  a  PC: 

0 

Used  a  PC: 

1 

Used  a  computer: 

1 

Languages: 

BASIC,  PYXEL  VISUALS 

Used  a  Macintosh: 

0 

House  Score: 

20 

House  Feature 

Walls 

Shape 

Texture 

Border 

Ridgeline 
Shape 
Texture 
Border 

Small  window 
Shape 
Texture 
Border 
Crosspieces 

Picture  window 
Shape 
Texture 
Border 


Door 


Score 

1 
1 


Shape 
Texture 
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Border  1 

Door  window 
Shape 
Texture 
Border 

Door  knob 
Shape 
Texture 

Roof 

Shape  1 

Texture  1 

Border  1 

Chimney 

Shape  1 

Texture 

Border  1 

Smoke 

Use  of  Pull-Down  Menu 

Texture 


Total  20 
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Subject  Number 
Sex:  M        Age: 

9 
32          Years  of  School: 

19 

Likes  games: 

1 

Types: 
Video  Games: 

Bridge 
0 

Comfortable  with  Math: 

1 

Math  Courses: 

Arithme 

[ic,  Algebra,  Geometry,  Trig, 

Calculus 

,  Logic,  Statistics 

Musical  Instrument: 

1 

Typing  Skills 

Typewriter: 

Speed: 

Word  Processor: 

1 
30 

1 

Own  a  PC: 

0 

Used  a  PC: 

1 

Used  a  computer: 
Languages: 

1 
BASIC, 

FORTRAN,  SPSS,  SAS 

Used  a  Macintosh: 

0 

House  Score: 

23 

House  Feature 

Score 

Walls 

Shape 
Texture 

1 
1 

Border 

1 

Ridgeline 

Shape 
Texture 

1 
1 

Border 

Small  window 

Shape 
Texture 

1 
1 

Border 

1 

Crosspieces 

1 

Picture  window 

Shape 
Texture 

1 

1 

Border 

1 

Door 

Shape 
Texture 

1 
1 

58 


Border  1 

Door  window 

Shape  1 

Texture  1 

Border  1 

Door  knob 
Shape 
Texture 

Roof 

Shape  1 

Texture  1 

Border 

Chimney 

Shape  1 

Texture  1 

Border  1 

Smoke 

Use  of  Pull-Down  Menu 
Texture 


Total  23 
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Subject  Number:    10 


Sex:    M       Age: 

45          Years  of  School: 
Likes  games: 
Types: 
Video  Games: 

16 
0 

0 

Comfortable  with  Math: 

0 

Math  Courses: 

Ari 

thmetic,  Algebra,  Geometry,  Trig 

Statistics 

I 

Vlusical  Instrument: 

0 

Typing  Skills 

Typewriter: 

Speed: 

Word  Processor: 

Own  a  PC: 

Used  a  PC: 

Used  a  computer: 

1 

30 

1 

0 

1 

1 

Languages: 

FORTRAN,  COBOL,  PL/I,  RPG,  A 

Used  a  Macintosh: 

0 

House  Score: 

7 

House  Feature 

Sco 

re 

Walls 

Shape 

Texture 

1 

Border 

Ridgeline 

Shape 

Texture 

Border 

Small  window 

Shape 

Texture 

1 

Border 

Crosspieces 

1 

Picture  window 

Shape 

Texture 

1 

Border 

Door 

Shape 

Texture 

1 

60 


Border 

Door  window 
Shape 
Texture 
Border 

Door  knob 
Shape 
Texture 

Roof 

Shape 

Texture 

Border 

Chimney 
Shape 
Texture 
Border 

Smoke 

Use  of  Pull-Down  Menu 
Texture 


Total 


Notes  -  the  subject  demonstrated  a  lack  of  interest  in  exploration  in  the  experiment 
that  is  consistent  with  his  professional  work  habits. 
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Subject  Number:    1 1 

Sex:    F       Age:  26          Years  of  School: 

IS 

Likes  games: 

1 

Types: 
Video  Games: 

Board  games 
1 

Comfortable  with  Math: 

0 

Math  Courses: 

Arithmetic,  Algebra,  Geometry,  Tri 
Calculus,  Statistics 

Musical  Instrument: 

1 

Typing  Skills 

Typewriter: 

Speed: 

Word  Processor: 

1 
30 

1 

Own  a  PC: 

1 

Used  a  PC: 

1 

Used  a  computer: 
Languages: 

1 
BASIC 

Used  a  Macintosh: 

0 

House  Score: 

24 

House  Feature 


Score 


Walls 

Shape 

Texture 

Border 

Ridgeline 
Shape 
Texture 
Border 

Small  window 
Shape 
Texture 
Border 
Crosspieces 

Picture  window 
Shape 
Texture 
Border 


Door 


Shape 
Texture 


Border 

Door  window 
Shape 
Texture 
Border 

Door  knob 
Shape 
Texture 

Roof 

Shape 

Texture 

Border 

Chimney 
Shape 
Texture 
Border 

Smoke 

Use  of  Pull-Down  Menu 
Texture 


Total 


62 


24 


63 


Subject  Number:    12 
Sex:    M       Age:  68          Years  of  School: 

Likes  games: 

Types: 

Video  Games: 

Comfortable  with  Math: 

Math  Courses: 

16 
0 

0 
0 

Ai 

Musical  Instrument: 
Typing  Skills 

Typewriter: 

Speed: 

Word  Processor: 

Own  a  PC: 

Used  a  PC: 

Used  a  computer: 

Languages: 

Used  a  Macintosh: 

0 

0 
0 
0 
0 
0 
o 

0 

House  Score: 

12 

House  Feature 

Walls 

Shape 

Texture 

Border 

Ridgeline 
Shape 
Texture 
Border 

Small  window 
Shape 
Texture 
Border 

Crosspieces 

Picture  window 
Shape 
Texture 
Border 

Door 

Shape 
Texture 

Arithmetic,  Algebra,  Geometry,  Trig 


Score 


64 


Border 

Door  window 
Shape 

Texture  1 

Border 

Door  knob 
Shape 
Texture  1 

Roof 

Shape 

Texture  1 

Border 

Chimney 

Shape  1 

Texture  1 

Border  1 

Smoke 

Use  of  Pull-Down  Menu 

Texture  1 


Total  12 


65 


Subject  Number 
Sex:    M      Age: 

:    13 

51      Years  of  School: 

20 

Likes  games: 

1 

Types: 
Video  Games: 

Chess,  Bridge 
0 

Comfortable  with  Math: 

0 

Math  Courses: 

Arithmetic,  Algebra,  Geometry,  Trig, 
Calculus,  Logic,  Statistics 

Musical  Instrument: 

1 

Typing  Skills 

Typewriter: 

Speed: 

Word  Processor: 

1 
0 

Own  a  PC: 

0 

Used  a  PC: 

0 

Used  a  computer: 
Languages: 

1 

FORTRAN 

Used  a  Macintosh: 

0 

House  Score: 

18 

House  Feature 

Score 

Walls 

Shape 
Texture 

Border 

Ridgeline 
Shape 
Texture 

Border 

Small  window 

Shape 

Texture 

Border 

Crosspieces 

Picture  window 

Shape 
Texture 

Border 

Door 

Shape 
Texture 

66 


Border  1 

Door  window 
Shape 

Texture  1 

Border 

Door  knob 
Shape 
Texture  1 

Roof 

Shape  1 

Texture  1 

Border 

Chimney 

Shape  1 

Texture  1 

Border 

Smoke 

Use  of  Pull-Down  Menu 
Texture 


Total  18 


Notes  -  Subject  is  reluctant  to  explore.   Tries  to  find  function  by  thinking 


67 


Subject  Number:    14 
Sex:    M      Age:    15 


Years  of  School 

Likes  games: 

Types: 

Video  Games: 

Comfortable  with  Math: 

Math  Courses: 


Typing  Skills 


Musical  Instrument: 

Typewriter: 

Speed: 

Word  Processor: 

Own  a  PC: 

Used  a  PC: 

Used  a  computer: 

Languages: 

Used  a  Macintosh: 


to 


1 

Chess 

I 

0 

Arithmetic,  Algebra 

0 

1 

15 

I 

0 

1 

0 

BASIC,  LOGO 

0 


House  Score: 


27 


House  Feature 


Score 


Walls 

Shape 

Texture 

Border 

Ridgeline 
Shape 
Texture 
Border 

Small  window 
Shape 
Texture 
Border 
Crosspieces 

Picture  window 
Shape 
Texture 
Border 


Door 


Shape 
Texture 


68 


Border  1 

Door  window 

Shape  1 

Texture  1 

Border  1 

Door  knob 

Shape  1 

Texture  1 

Roof 

Shape  1 

Texture  1 

Border  1 

Chimney 

Shape  1 

Texture  1 

Border  1 

Smoke 

Use  of  Pull-Down  Menu 
Texture 


Total  27 


69 


Subject  Number:    15 

Sex:  F        Age:    38        Years  of  School: 

14 

Likes  games: 

0 

Types: 
Video  Games: 

0 

Comfortable  with  Math: 

1 

Math  Courses: 

Arithmetic,  Algebra, 

Musical  Instrument: 

Geometry 
0 

Typing  Skills 

Typewriter: 

Speed: 

Word  Processor: 

0 

70 

1 

Own  a  PC: 

0 

Used  a  PC: 

1 

Used  a  computer: 
Languages: 

1 
BASIC 

Used  a  Macintosh: 

0 

House  Score: 

10 

House  Feature 

Walls 

Shape 

Texture 

Border 

Ridgeline 
Shape 
Texture 
Border 

Small  window 
Shape 
Texture 
Border 
Crosspieces 

Picture  window 
Shape 
Texture 
Border 


Score 


Door 


Shape 
Texture 


70 


Border 

Door  window 

Shape  1 

Texture  1 

Border 

Door  knob 
Shape 
Texture 

Roof 

Shape 

Texture 

Border 

Chimney 

Shape  1 

Texture 

Border 

Smoke 

Use  of  Pull-Down  Menu 
Texture 


Total 


71 


Subject  Numbe 
Sex:  M       Age: 

r:    16 
36       Years  of  School: 

14 

- 

Likes  games: 

Types: 

Video  Games: 

1 

Poker 
0 

Comfortable  with  Math: 

0 

Math  Courses: 

Arithmetic,  Algebra 

Musical  Instrument: 

0 

Typing  Skills 

Typewriter: 

Speed: 

Word  Processor: 

1 

so 

1 

Own  a  PC: 

0 

Used  a  PC: 

o 

Used  a  computer: 

0 

Languages: 

Used  a  Macintosh: 

0 

House  Score: 

II 

House  Feature 

Score 

Walls 

Shape 
Texture 

1 

Border 

1 

Ridgeline 

Shape 
Texture 

Border 

Small  window 

Shape 
Texture 

1 

1 

Border 

1 

Crosspieces 

Picture  window 

Shape 
Texture 

1 
1 

Border 

Door 

Shape 
Texture 

1 
1 

72 


Border 

Door  window 
Shape 
Texture 
Border 

Door  knob 
Shape 
Texture 


Shape  ! 

e 

1 


Roof 

Texture 
Border 

Chimney 

Shape 

Texture 

Border 

Smoke 

Use  of  Pull-Down  Menu 

Texture 


Total  [  l 


73 


Subject  Number:    17 


Sex:    M      Age: 

45      Years  of  School: 

16 

Likes  games: 

0 

Types: 
Video  Games: 

0 

Comfortable  with  Math: 

0 

Math  Courses: 

Arithmetic,  Algebra,  Geometry, 
Calculus,  Logic,  Statistics 

Trig, 

Vlusical  Instrument: 

0 

Typing  Skills 

Typewriter: 

Speed: 

Word  Processor: 

1 

30 

1 

Own  a  PC: 

0 

Used  a  PC: 

1 

Used  a  computer: 
Languages: 

1 

BASIC,  PASCAL,  FORTRAN, 

PL/ 1 

Used  a  Macintosh: 

0 

House  Score: 

3 

Mouse  Feature 

Score 

Walls 

Shape 
Texture 

1 
1 

Border 

Ridgeline 

Shape 
Texture 

Border 

Small  window 

Shape 
Texture 

Border 

Crosspieces 

Picture  window 

Shape 
Texture 

Border 

Door 

Shape 
Texture 

74 


Border 

Door  window 
Shape 
Texture 
Border 

Door  knob 
Shape 
Texture 

Roof 

Shape 

Texture 

Border 

Chimney 

Shape 

Texture 

Border 

Smoke 

Use  of  Pull-Down  Menu 
Texture 


Total 


~ 


Subject  Number:    1! 


Sex:  M       Age: 

52      Years  of  School: 

17 

Likes  games: 

0 

Types: 
Video  Games: 

0 

Comfortable  with  Math: 

0 

Math  Courses: 

Arithmei 

:ic,  Algebra,  Geometry,  Trig, 

Calculus 

,  Logic 

Musical  Instrument: 

0 

Typing  Skills 

Typewriter: 

Speed: 

Word  Processor: 

I 

30 

0 

Own  a  PC: 

0 

Used  a  PC: 

1 

Used  a  computer: 
Languages: 

1 
BASIC, 

PASCAL,  FORTRAN,  ASS 

Used  a  Macintosh: 

0 

House  Score: 

[5 

House  Feature 

Score 

Walls 

Shape 
Texture 

1 

1 

Border 

Ridgeline 

Shape 
Texture 

1 

Border 

Small  window 

Shape 
Texture 

1 
1 

Border 

Crosspieces 

1 

Picture  window 

Shape 
Texture 

1 

1 

Border 

Door 

Shape 
Texture 

1 
1 

76 


Border 

Door  window 

Shape  1 

Texture  1 

Border 

Door  knob 
Shape 
Texture 

Roof 

Shape  1 

Texture 

Border 

Chimney 

Shape  1 

Texture  1 

Border 

Smoke 

Use  of  Pull-Down  Menu 
Texture 


Total  15 


" 


Subject  Number 

19 

Sex:    M       Age: 

36          Years  of  School: 

20 

Likes  games: 

1 

Types: 

Chess,  B 

ridge,  Go 

Video  Games: 

1 

Comfortable  with  Math: 

0 

Math  Courses: 

Arithmetic,  Algebra,  Geometry,  Tri 

Calculus 

,  Logic,  Statistics 

Musical  Instrument: 

1 

Typing  Skills 

Typewriter: 

1 

Speed: 

35 

Word  Processor: 

1 

Own  a  PC: 

1 

Used  a  PC: 

1 

Used  a  computer: 

1 

Languages: 

BASIC, 

FORTRAN 

Used  a  Macintosh: 

0 

House  Score: 

19 

House  Feature 

Score 

Walls 

Shape 

1 

Texture 

Border 

1 

Ridgeline 

Shape 

1 

Texture 

Border 

1 

Small  window 

Shape 

1 

Texture 

1 

Border 

1 

Crosspieces 

1 

Picture  window 

Shape 

1 

Texture 

1 

Border 

1 

Door 

Shape 

1 

Texture 

1 

78 


Border  1 

Door  window 

Shape  .                                                              1 

Texture  1 

Border  1 

Door  knob 
Shape 
Texture 

Roof 

Shape  1 

Texture 

Border  1 

Chimney 
Shape 
Texture 
Border 

Smoke 

Use  of  Pull- Down  Menu 
Texture 


Total  19 


~> 


Subject  Number: 

20  Sex:  F        Age:  7 

Years  of  Scho 

Likes  games: 

1 

Types: 
Video  Games: 

1 

Comfortable  with  Math: 

1 

Math  Courses: 

Arithmetic 

Musical  Instrument: 

1 

Typing  Skills 

Typewriter: 

Speed: 

Word  Processor: 

i) 
0 
0 

Own  a  PC: 

1 

Used  a  PC: 

1 

Used  a  computer: 

1 

Languages: 

I 

Jsed  a  Macintosh: 

0 

House  Score: 

s 

House  Feature 

Score 

Walls 

Shape 
Texture 

Border 

Ridgeline 
Shape 
Texture 

Border 

Small  window 

Shape 
Texture 

1 
1 

Border 

Crosspieces 

1 

Picture  window 

Shape 
Texture 

1 

Border 

Door 

Shape 
Texture 

1 

Border 

80 


Door  window 
Shape 
Texture 
Border 

Door  knob 
Shape 
Texture 

Roof 

Shape 

Texture 

Border 

Chimney 
Shape 
Texture 
Border 

Smoke 

Use  of  Pull-Down  Menu 
Texture 


Total 
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